Pith. sign in

REVIEW 1 major objections 1 minor 55 references

Reviewed by Pith at T0; open to challenge.

T0 means a machine referee read the full paper against a public rubric. The mark states how deep the mechanical check went, never who wrote it. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.3

Representative Attention compresses vision transformer tokens in representation space to achieve linear global attention.

2026-06-30 20:52 UTC pith:R7L5RHEC

load-bearing objection RPAttention moves token compression into representation space via competitive routing, but the abstract leaves the stability of that routing and the actual results unshown. the 1 major comments →

arxiv 2605.14913 v1 pith:R7L5RHEC submitted 2026-05-14 cs.CV

Representative Attention For Vision Transformers

Yuntong Li , Hainuo Wang , Hengxing Liu , Mingjia Li , Xiaojie Guo This is my paper
classification cs.CV
keywords representative attentionlinear attentionvision transformerstoken compressionsemantic routingglobal contextgather interact distribute
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces Representative Attention to address the quadratic cost of self-attention in vision transformers by compressing tokens into a compact set of learned representatives. Unlike prior linear attention methods that rely on fixed spatial partitions, this approach gathers tokens through competitive similarity routing directly in feature space. The representatives then interact globally before distributing refined information back to spatial tokens. This design aims to align token communication with semantic content rather than image coordinates while preserving global receptive fields. Experiments on classification, detection, and segmentation tasks across multiple backbones test whether the method maintains performance at linear scaling.

Core claim

RPAttention is a linear global attention mechanism that performs token compression directly in representation space by dynamically forming a compact set of learned representative tokens through competitive similarity-based routing; these representatives enable semantically related regions to communicate independent of spatial distance via a Gather-Interact-Distribute paradigm before broadcasting information back through query-driven cross-attention, thereby reducing dominant token interaction complexity from quadratic to linear scaling while maintaining expressive global context modeling.

What carries the argument

Gather-Interact-Distribute paradigm with competitive similarity-based routing into a fixed set of representative tokens that mediate global interaction in a compact latent space

Load-bearing premise

Competitive similarity-based routing into a fixed number of representative tokens will stably capture the semantic organization of visual content across diverse inputs without substantial information loss or training instability.

What would settle it

A direct comparison on a dataset of images with strong semantic relations across distant spatial locations, measuring whether RPAttention maintains accuracy parity with quadratic attention while using the same number of parameters and training steps.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Token interactions align with semantic similarity rather than predefined spatial layouts
  • Global context modeling is preserved at linear rather than quadratic cost in the number of spatial tokens
  • The mechanism integrates into existing vision transformer backbones without altering their overall architecture
  • Effectiveness holds across image classification, object detection, and semantic segmentation tasks

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The representation-driven routing may generalize to sequences where positional structure is less predictive than content similarity
  • Adaptive choice of representative count per input could further reduce compute on simple scenes
  • Cross-attention broadcast step may introduce bottlenecks if representative count grows with input complexity

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

1 major / 1 minor

Summary. The manuscript proposes Representative Attention (RPAttention), a linear-complexity global attention mechanism for Vision Transformers. It replaces coordinate-driven token compression with representation-driven compression via a Gather-Interact-Distribute paradigm: spatial tokens are softly aggregated into a fixed set of learned representative tokens through competitive similarity-based routing, the representatives perform global interaction in a compact latent space, and refined information is broadcast back to spatial tokens via query-driven cross-attention. The method is claimed to enable semantically related regions to interact independently of spatial distance, reduce dominant token-interaction complexity from quadratic to linear in the number of spatial tokens, and maintain expressive global context; effectiveness is demonstrated via experiments on image classification, object detection, and semantic segmentation across multiple ViT backbones.

Significance. If the empirical claims hold, the work would address a recognized limitation of prior linear-attention proxies (their dependence on fixed spatial partitions) by making compression content-adaptive in representation space. This could improve both efficiency and semantic fidelity in large-scale vision models. The linear scaling itself is not novel, but the explicit separation of routing from image coordinates is a targeted refinement that, if stable, would be a useful incremental contribution.

major comments (1)
  1. [Abstract] Abstract (and the Gather step description): the claim that competitive similarity-based routing 'dynamically forms a compact set of learned representative tokens' to align with semantic organization is load-bearing for both the semantic-alignment benefit and the linear-complexity guarantee. No mechanism (temperature annealing, auxiliary loss, initialization strategy, or collapse-prevention term) is specified to ensure the soft assignments remain stable across diverse inputs or that distinct semantics are not merged when N tokens are compressed to K representatives. If routing collapses or defaults to spatial proximity, the subsequent Interact and Distribute steps cannot deliver the claimed global context modeling.
minor comments (1)
  1. The abstract states that 'extensive experiments across diverse vision transformer backbones' demonstrate effectiveness, yet supplies no quantitative results, baselines, ablations, or error bars. This omission prevents assessment of whether the reported gains are statistically meaningful or merely consistent with existing linear-attention methods.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the careful review and constructive feedback. We respond to the major comment below.

read point-by-point responses

  1. Referee: [Abstract] Abstract (and the Gather step description): the claim that competitive similarity-based routing 'dynamically forms a compact set of learned representative tokens' to align with semantic organization is load-bearing for both the semantic-alignment benefit and the linear-complexity guarantee. No mechanism (temperature annealing, auxiliary loss, initialization strategy, or collapse-prevention term) is specified to ensure the soft assignments remain stable across diverse inputs or that distinct semantics are not merged when N tokens are compressed to K representatives. If routing collapses or defaults to spatial proximity, the subsequent Interact and Distribute steps cannot deliver the claimed global context modeling.

    Authors: We agree that the stability of the competitive similarity-based routing is essential to the semantic-alignment claim. The provided manuscript text does not specify mechanisms such as temperature annealing, auxiliary losses, or explicit collapse-prevention terms. The linear complexity guarantee holds via the fixed number of representatives K regardless of routing behavior, but the semantic benefit does depend on non-collapsed assignments. We will revise the manuscript to expand the Gather-step description with the exact similarity computation and hyperparameters used, and we will add an empirical analysis (e.g., visualization of assignment patterns or an ablation on routing behavior) to show that assignments align with semantics rather than spatial proximity on the evaluated datasets. revision: yes

Circularity Check

0 steps flagged

No circularity: proposed architecture with direct complexity reduction

full rationale

The paper presents RPAttention as a new Gather-Interact-Distribute mechanism that replaces coordinate-driven token aggregation with representation-driven compression via competitive similarity routing. No equations, derivations, or predictions are shown that reduce the claimed linear scaling or semantic alignment benefit to a fitted quantity defined by the method itself. No self-citations, uniqueness theorems, or ansatzes imported from prior work appear in the abstract or description. The complexity claim follows directly from performing interactions in a compact K-token space rather than N tokens, which is an architectural property rather than a circular redefinition. The derivation chain is self-contained as an empirical design proposal.

Axiom & Free-Parameter Ledger

1 free parameters · 0 axioms · 0 invented entities

Abstract-only review; no explicit free parameters, axioms, or invented entities are stated. The number of representative tokens is an implicit design choice whose value is not provided.

free parameters (1)
  • number of representative tokens
    Compact set size is a core design parameter required for the compression step but not quantified in the abstract.

pith-pipeline@v0.9.1-grok · 5782 in / 1106 out tokens · 23013 ms · 2026-06-30T20:52:39.367654+00:00 · methodology

0 comments
read the original abstract

Linear attention has emerged as a promising direction for scaling Vision Transformers beyond the quadratic cost of dense self-attention. A prevalent strategy is to compress spatial tokens into a compact set of intermediate proxies that mediate global information exchange. However, existing methods typically derive these proxy tokens from predefined spatial layouts, causing token compression to remain anchored to image coordinates rather than the semantic organization of visual content. To overcome this limitation, we propose Representative Attention (RPAttention), a linear global attention mechanism that performs token compression directly in representation space. Instead of constructing intermediate tokens from fixed spatial partitions, it dynamically forms a compact set of learned representative tokens to enable semantically related regions to communicate regardless of their spatial distance, by following a lightweight Gather-Interact-Distribute paradigm. Spatial tokens are first softly gathered into representative tokens through competitive similarity-based routing. The representatives then perform global interaction within a compact latent space, before broadcasting the refined information back to all spatial tokens via query-driven cross-attention. Via replacing coordinate-driven aggregation with representation-driven compression, RPAttention preserves global receptive fields while adaptively aligning token communication with the content structure of each input.RPAttention reduces the dominant token interaction complexity from quadratic to linear scaling with respect to the number of spatial tokens, while maintaining expressive global context modeling. Extensive experiments across diverse vision transformer backbones on image classification, object detection, and semantic segmentation demonstrate the effectiveness of our design.

Figures

Figures reproduced from arXiv: 2605.14913 by Hainuo Wang, Hengxing Liu, Mingjia Li, Xiaojie Guo, Yuntong Li.

Figure 1
Figure 1. Figure 1: Motivation of Representative Attention. (a) An input image with three query locations [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of Representative Attention. Spatial tokens are gathered into a compact set of [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Analysis of representative token construction. Left: translation robustness under shifted [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative limitation case. SAtten and RPAtten denote spatial attention and Representative [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

55 extracted references · 55 canonical work pages · 2 internal anchors

  1. [1]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. InICLR, 2021

    work page 2021

  2. [2]

    Pyramid vision transformer: A versatile backbone for dense prediction without convolutions

    Wenhai Wang, Enze Xie, Xiang Li, Deng-Ping Fan, Kaitao Song, Ding Liang, Tong Lu, Ping Luo, and Ling Shao. Pyramid vision transformer: A versatile backbone for dense prediction without convolutions. InICCV, 2021

    work page 2021

  3. [3]

    PVT v2: Improved baselines with pyramid vision transformer

    Wenhai Wang, Enze Xie, Xiang Li, Deng-Ping Fan, Kaitao Song, Ding Liang, Tong Lu, Ping Luo, and Ling Shao. PVT v2: Improved baselines with pyramid vision transformer. Computational Visual Media, 2022

    work page 2022

  4. [4]

    Twins: Revisiting the design of spatial attention in vision transformers

    Xiangxiang Chu, Zhi Tian, Yuqing Wang, Bo Zhang, Haibing Ren, Xiaolin Wei, Huaxia Xia, and Chunhua Shen. Twins: Revisiting the design of spatial attention in vision transformers. In NeurIPS, 2021

    work page 2021

  5. [5]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. InICCV, 2021

    work page 2021

  6. [6]

    Neighborhood attention transformer

    Ali Hassani, Steven Walton, Nikhil Shah, Abudukelimu Abuduweili, Jiachen Li, and Humphrey Shi. Neighborhood attention transformer. InCVPR, 2023

    work page 2023

  7. [7]

    CSWin transformer: A general vision transformer backbone with cross-shaped windows

    Xiaoyi Dong, Jianmin Bao, Dongdong Chen, Weiming Zhang, Nenghai Yu, Lu Yuan, Dong Chen, and Baining Guo. CSWin transformer: A general vision transformer backbone with cross-shaped windows. InCVPR, 2022

    work page 2022

  8. [8]

    Scaling local self-attention for parameter efficient visual backbones

    Ashish Vaswani, Prajit Ramachandran, Aravind Srinivas, Niki Parmar, Blake Heathershaw, and Noam Shazeer. Scaling local self-attention for parameter efficient visual backbones. InCVPR, 2021

    work page 2021

  9. [9]

    Transformers are RNNs: Fast autoregressive transformers with linear attention

    Angelos Katharopoulos, Apoorv Vyas, Nikolaos Pappas, and François Fleuret. Transformers are RNNs: Fast autoregressive transformers with linear attention. InICML, 2020

    work page 2020

  10. [10]

    Rethinking attention with performers

    Krzysztof Choromanski, Valerii Llowne, Anton Cassel, Tamas Sarlos, Anne Adrian, et al. Rethinking attention with performers. InICLR, 2021

    work page 2021

  11. [11]

    cosFormer: Rethinking Softmax in attention

    Zhen Qin, Weixuan Sun, Hui Deng, Dongxu Li, Yunshen Wei, Baohong Lv, Jun Yan, Lingpeng Kong, and Yiran Zhong. cosFormer: Rethinking Softmax in attention. InICLR, 2022

    work page 2022

  12. [12]

    PolaFormer: Polarity- aware linear attention for vision transformers

    Weikang Meng, Yadan Luo, Xin Li, Dongmei Jiang, and Zheng Zhang. PolaFormer: Polarity- aware linear attention for vision transformers. InICLR, 2025

    work page 2025

  13. [13]

    Linformer: Self-Attention with Linear Complexity

    Sinong Wang, Belinda Z Li, Madian Khabsa, Hanwen Fang, and Hao Ma. Linformer: Self- attention with linear complexity.arXiv preprint arXiv:2006.04768, 2020

    work page internal anchor Pith review Pith/arXiv arXiv 2006

  14. [14]

    Nyströmformer: A Nyström-based algorithm for approximating self-attention

    Yunyang Xiong, Zhanpeng Zeng, Rudrasis Chakraborty, Mingxing Tan, Glenn Fung, Yin Li, and Vikas Singh. Nyströmformer: A Nyström-based algorithm for approximating self-attention. InAAAI, 2021

    work page 2021

  15. [15]

    Vssd: Vision mamba with non-causal state space duality

    Yuheng Shi, Mingjia Li, Minjing Dong, and Chang Xu. Vssd: Vision mamba with non-causal state space duality. InICCV 2025, pages 10819–10829, 2025

    work page 2025

  16. [16]

    Modem: A morton-order degradation estimation mechanism for adverse weather image recovery

    Hainuo Wang, Qiming Hu, and Xiaojie Guo. Modem: A morton-order degradation estimation mechanism for adverse weather image recovery. InNeurIPS 2025, 2025. 10

    work page 2025

  17. [17]

    Ghostnet: More features from cheap operations

    Kai Han, Yunhe Wang, Qi Tian, Jianyuan Guo, Chunjing Xu, and Chang Xu. Ghostnet: More features from cheap operations. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1580–1589, 2020

    work page 2020

  18. [18]

    Dynam- icViT: Efficient vision transformers with dynamic token sparsification

    Yongming Rao, Wenliang Zhao, Benlin Liu, Jiwen Lu, Jie Zhou, and Cho-Jui Hsieh. Dynam- icViT: Efficient vision transformers with dynamic token sparsification. InAdvances in Neural Information Processing Systems (NeurIPS), volume 34, pages 13937–13949, 2021

    work page 2021

  19. [19]

    Token merging: Your ViT but faster

    Daniel Bolya, Cheng-Yang Fu, Xiaoliang Dai, Peizhao Zhang, Christoph Feichtenhofer, and Judy Hoffman. Token merging: Your ViT but faster. InInternational Conference on Learning Representations (ICLR), 2023

    work page 2023

  20. [20]

    Not all patches are what you need: Expediting vision transformers via token reorganizations

    Youwei Liang, Chongjian Ge, Zhan Tong, Yang Song, Jue Wang, and Pengtao Xie. Not all patches are what you need: Expediting vision transformers via token reorganizations. InICLR, 2022

    work page 2022

  21. [21]

    Agent attention: On the integration of softmax and linear attention

    Dongchen Han, Tianzhu Ye, Yizeng Han, Zhuofan Xia, Shiji Song, and Gao Huang. Agent attention: On the integration of softmax and linear attention. InECCV, 2024

    work page 2024

  22. [22]

    Emerging properties in self-supervised vision transformers

    Mathieu Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. InICCV, 2021

    work page 2021

  23. [23]

    Wit: Waypoint diffusion transformers via trajectory conflict navigation.arXiv preprint arXiv:2603.15132, 2026

    Hainuo Wang*, Mingjia Li*, and Xiaojie Guo. Wit: Waypoint diffusion transformers via trajectory conflict navigation.arXiv preprint arXiv:2603.15132, 2026

    work page arXiv 2026

  24. [24]

    Regional attention for shadow removal

    Hengxing Liu, Mingjia Li, and Xiaojie Guo. Regional attention for shadow removal. InACM Multimedia 2024, pages 5949–5957. 2024

    work page 2024

  25. [25]

    On the Global Photometric Alignment for Low-Level Vision

    Mingjia Li, Tianle Du, Hainuo Wang, Qiming Hu, and Xiaojie Guo. On the global photometric alignment for low-level vision.arXiv preprint arXiv:2604.08172, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  26. [26]

    Shadowhack: Hacking shadows via luminance-color divide and conquer.ICCV 2025, 2024

    Jin Hu*, Mingjia Li*, and Xiaojie Guo. Shadowhack: Hacking shadows via luminance-color divide and conquer.ICCV 2025, 2024

    work page 2025

  27. [27]

    Object-centric learning with slot attention

    Francesco Locatello, Dirk Weissenborn, Thomas Unterthiner, Aravindh Mahendran, Georg Heigold, Jakob Uszkoreit, Alexey Dosovitskiy, and Thomas Kipf. Object-centric learning with slot attention. InNeurIPS, 2020

    work page 2020

  28. [28]

    Single image reflection separation via dual-stream interactive transformers.Advances in Neural Information Processing Systems, 37:55228–55248, 2024

    Qiming Hu, Hainuo Wang, and Xiaojie Guo. Single image reflection separation via dual-stream interactive transformers.Advances in Neural Information Processing Systems, 37:55228–55248, 2024

    work page 2024

  29. [29]

    Training data-efficient image transformers & distillation through attention

    Hugo Touvron, Matthieu Cord, Matthijs Douze, Francisco Massa, Alexandre Sablayrolles, and Hervé Jégou. Training data-efficient image transformers & distillation through attention. In ICML, 2021

    work page 2021

  30. [30]

    Focal self-attention for local-global interactions in vision transformers

    Jianwei Yang, Chunyuan Li, Pengchuan Zhang, Xiaodong Dai, Bin Xiao, Lu Yuan, and Jianfeng Gao. Focal self-attention for local-global interactions in vision transformers. InNeurIPS, 2021

    work page 2021

  31. [31]

    Reversible decoupling network for single image reflection removal

    Hao Zhao*, Mingjia Li*, Qiming Hu, and Xiaojie Guo. Reversible decoupling network for single image reflection removal. InCVPR 2025, pages 26430–26439, 2025

    work page 2025

  32. [32]

    Fast vision transformers with HiLo attention

    Zizheng Pan, Jianfei Cai, and Bohan Zhuang. Fast vision transformers with HiLo attention. In NeurIPS, 2022

    work page 2022

  33. [33]

    TransNeXt: Robust foveal visual perception for vision transformers

    Dai Shi. TransNeXt: Robust foveal visual perception for vision transformers. InCVPR, 2024

    work page 2024

  34. [34]

    VMamba: Visual state space model

    Yue Liu, Yunjie Tian, Yuzhong Zhao, Hongtian Yu, Lingxi Xie, Yaowei Wang, Qixiang Ye, Jianbin Jiao, and Yunfan Liu. VMamba: Visual state space model. InNeurIPS, 2024

    work page 2024

  35. [35]

    MambaVision: A hybrid Mamba-transformer vision backbone

    Ali Hatamizadeh and Jan Kautz. MambaVision: A hybrid Mamba-transformer vision backbone. InCVPR, 2025

    work page 2025

  36. [36]

    Vision-TTT: Efficient and expressive visual representation learning with test-time training.arXiv preprint arXiv:2603.00518, 2026

    Quan Kong, Yanru Xiao, Yuhao Shen, and Cong Wang. Vision-TTT: Efficient and expressive visual representation learning with test-time training.arXiv preprint arXiv:2603.00518, 2026. 11

    work page arXiv 2026

  37. [37]

    Breaking the low-rank dilemma of linear attention

    Qihang Fan, Huaibo Huang, and Ran He. Breaking the low-rank dilemma of linear attention. In CVPR, 2025

    work page 2025

  38. [38]

    CARE transformer: Mobile-friendly linear visual transformer via decoupled dual interaction

    Yuan Zhou, Qingshan Xu, Jiequan Cui, Junbao Zhou, Jing Zhang, Richang Hong, and Hanwang Zhang. CARE transformer: Mobile-friendly linear visual transformer via decoupled dual interaction. InCVPR, 2025

    work page 2025

  39. [39]

    Vision transformer with deformable attention

    Zhuofan Xia, Xuran Pan, Shiji Song, Haosen Li, and Gao Huang. Vision transformer with deformable attention. InCVPR, 2022

    work page 2022

  40. [40]

    Lei Zhu, Xinjiang Wang, Zhanghan Ke, Wayne Zhang, and Rynson W. H. Lau. BiFormer: Vision transformer with bi-level routing attention. InCVPR, pages 10323–10333, 2023

    work page 2023

  41. [41]

    Rethinking token reduction with parameter- efficient fine-tuning in ViT for pixel-level tasks

    Cheng Lei, Ao Li, Hu Yao, Ce Zhu, and Le Zhang. Rethinking token reduction with parameter- efficient fine-tuning in ViT for pixel-level tasks. InCVPR, 2025

    work page 2025

  42. [42]

    Dynamic routing between capsules

    Sara Sabour, Nicholas Frosst, and Geoffrey E Hinton. Dynamic routing between capsules. In NeurIPS, 2017

    work page 2017

  43. [43]

    STEGO: Unsupervised semantic segmentation by distilling feature correspondences

    Mark Hamilton, Zhoutong Zhang, Bharath Hariharan, Noah Snavely, and William T Freeman. STEGO: Unsupervised semantic segmentation by distilling feature correspondences. InICLR, 2022

    work page 2022

  44. [44]

    Perceiver IO: A general architecture for structured inputs & outputs

    Andrew Jaegle, Sebastian Borgeaud, Jean-Baptiste Alayrac, Carl Doersch, Catalin Ionescu, et al. Perceiver IO: A general architecture for structured inputs & outputs. InICLR, 2022

    work page 2022

  45. [45]

    GroupViT: Semantic segmentation emerges from text supervision

    Jiarui Xu, Shalini De Mello, Sifei Liu, Wonmin Yin, Jan Kautz, and Xiaolong Wang. GroupViT: Semantic segmentation emerges from text supervision. InCVPR, 2022

    work page 2022

  46. [46]

    k-means Mask Trans- former

    Qihang Yu, Huiyu Wang, Jiawen Pan, Chunhua Shen, and Alan Yuille. k-means Mask Trans- former. InECCV, 2022

    work page 2022

  47. [47]

    FLatten transformer: Vision transformer using focused linear attention

    Dongchen Han, Xuran Pan, Yizeng Han, Shiji Song, and Gao Huang. FLatten transformer: Vision transformer using focused linear attention. InICCV, pages 5961–5971, 2023

    work page 2023

  48. [48]

    ImageNet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A large-scale hierarchical image database. InCVPR, 2009

    work page 2009

  49. [49]

    Lawrence Zitnick

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. Microsoft COCO: Common objects in context. InECCV, 2014

    work page 2014

  50. [50]

    Semantic understanding of scenes through the ADE20K dataset.International Journal of Computer Vision, 2019

    Bolei Zhou, Hang Zhao, Xavier Puig, Tete Xiao, Sanja Fidler, Adela Barriuso, and Antonio Torralba. Semantic understanding of scenes through the ADE20K dataset.International Journal of Computer Vision, 2019

    work page 2019

  51. [51]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. InICLR, 2019

    work page 2019

  52. [52]

    RandAugment: Practical automated data augmentation with a reduced search space

    Ekin D Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V Le. RandAugment: Practical automated data augmentation with a reduced search space. InNeurIPS, 2020

    work page 2020

  53. [53]

    mixup: Beyond empirical risk minimization

    Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. InICLR, 2018

    work page 2018

  54. [54]

    CutMix: Regularization strategy to train strong classifiers with localizable features

    Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, and Youngjoon Yoo. CutMix: Regularization strategy to train strong classifiers with localizable features. In ICCV, 2019

    work page 2019

  55. [55]

    Random erasing data augmentation.AAAI, 2020

    Zhun Zhong, Liang Zheng, Guoliang Kang, Shaozi Li, and Yi Yang. Random erasing data augmentation.AAAI, 2020. 12 A Appendix A.1 Additional Query-Response Visualizations Figure A.1: Additional query-response visualizations. For selected query locations, RPAttention produces response maps that align with semantically related image regions, illustrating how r...

    work page 2020