The reviewed record of science sign in
Pith

arxiv: 2604.18543 · v4 · pith:KPVMV4X4 · submitted 2026-04-20 · cs.AI · cs.CL

ClawEnvKit: Automatic Environment Generation for Claw-Like Agents

Xirui Li , Ming Li , Ion Stoica , Cho-Jui Hsieh , Tianyi Zhou This is my paper

Reviewed by Pith2026-07-05 11:26 UTCglm-5.2pith:KPVMV4X4open to challenge →

classification cs.AI cs.CL
keywords vision transformerspatial priorEuclidean distanceattention mechanismtoken groupingimage classificationobject detectionsemantic segmentation
0
0 comments X

The pith

Euclidean decay beats Manhattan in vision transformers

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper claims that replacing Manhattan distance with Euclidean distance in the spatial decay matrix of a vision transformer yields better image understanding, because human visual attention decays radially rather than along grid axes. The authors build EVT (Euclidean Enhanced Vision Transformer) on top of their prior RMT model, making two core changes: swapping the L1-based decay for an L2-based decay that provides smooth, rotationally invariant spatial priors, and replacing the decomposed horizontal-vertical attention mechanism with a simpler one-dimensional token grouping method that is faster and more flexible. The central argument is that Euclidean distance more naturally matches the geometry of images — its gradients are smooth and direction-aware, its spectral properties yield isotropic spatial harmonics, and its attention distribution more closely resembles what a trained ViT learns on its own. Combined with 1D grouping (which ignores spatial layout in how tokens are partitioned but compensates via the Euclidean decay matrix restoring spatial structure), EVT achieves 86.6% top-1 accuracy on ImageNet-1K at 384×384 resolution without additional training data, while maintaining higher inference throughput than its predecessor across all model scales.

Core claim

The paper establishes that a Euclidean (L2) distance-based spatial decay matrix, when applied directly to attention scores in a vision transformer, provides spatial priors that are both more effective and more stable than Manhattan (L1) distance. The authors support this with a Jensen-Shannon divergence analysis showing the Euclidean decay distribution more closely matches learned attention patterns, gradient analysis showing smooth direction-aware derivatives versus L1's piecewise-constant discontinuities, and spectral analysis showing isotropic spatial harmonics versus axis-aligned ones. They further show that a one-dimensional token grouping method — which flattens the spatial grid before

What carries the argument

Euclidean Self-Attention (EuSA): standard self-attention with a multiplicative decay matrix E where each entry E_nm = gamma^{sqrt((x_n - x_m)^2 + (y_n - y_m)^2)}, gamma being a per-head decay coefficient. Combined with 1D grouped (EuSA_g) and dilated (EuSA_d) variants that flatten the 2D token grid into a sequence before partitioning, alternating between the two to achieve global receptive field coverage.

If this is right

  • If radial distance decay is genuinely superior to grid-aligned decay, other spatial prior injection methods (relative position encodings, window attention biases) may benefit from replacing L1 or grid-based schemes with isotropic L2 formulations.
  • The finding that 1D grouping with a strong spatial decay prior matches or exceeds 2D grouping suggests that spatial structure can be decoupled from token partitioning — the decay matrix carries spatial knowledge while grouping handles computational efficiency.
  • The instability of higher-order distance functions (Minkowski p>=3, RBF kernels) implies there is a practical sweet spot in complexity for spatial priors: enough to encode geometry, simple enough to avoid training collapse.
  • EVT's consistent gains on out-of-distribution robustness benchmarks (ImageNet-A, ImageNet-R) suggest that explicit spatial priors may improve generalization beyond in-distribution accuracy.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The JS divergence argument for Euclidean over Manhattan is partially circular because the reference model (EVT-T without decay) still uses EVT's architectural choices including 1D grouping and CPE. A cleaner test would use a standard ViT or DeiT as the reference attention distribution.
  • The 0.3% accuracy gap between Manhattan and Euclidean decay on a single model size (EVT-T) may be within seed-to-seed variance; the claim would be stronger with multiple runs and error bars.
  • If the radial decay hypothesis is correct, one might predict even larger gains on tasks with strong rotational structure (e.g., medical imaging, aerial imagery) where axis-aligned biases would be most harmful.
  • The per-head varying gamma coefficient (gamma_n = 1 - 2^{-3-n}) functions as a multi-scale receptive field mechanism, but the specific formula appears heuristic; a learned or searched gamma schedule might yield further gains.

Load-bearing premise

The claim that Euclidean distance is superior to Manhattan distance rests on a JS divergence comparison where the reference model is itself an EVT variant, making the comparison partially circular, and the empirical accuracy gap of 0.3% between the two distance functions on a single model size may be within noise.

What would settle it

Train identical EVT architectures with Manhattan vs. Euclidean decay across multiple random seeds with error bars; if the 0.3% gap on EVT-T is not statistically significant, the core empirical claim weakens. Additionally, if a non-EVT reference model (e.g., standard DeiT or Swin) shows attention patterns closer to Manhattan decay than Euclidean, the distribution-similarity argument collapses.

read the original abstract

Constructing environments for training and evaluating claw-like agents remains a manual, human-intensive process that does not scale. We argue that what is needed is not just a dataset, but an automated pipeline capable of generating diverse, verified environments on demand. To this end, we introduce ClawEnvKit, an autonomous generation pipeline that instantiates this formalism from natural language descriptions. The pipeline comprises three modules: (1) a parser that extracts structured generation parameters from natural language input; (2) a generator that produces the task specification, tool interface, and scoring configuration; and (3) a validator that enforces feasibility, diversity, structural validity, and internal consistency across the generated environments. Using ClawEnvKit, we construct Auto-ClawEval, the first large-scale benchmark for claw-like agents, comprising 1,040 environments across 24 categories. Empirically, Auto-ClawEval matches or exceeds human-curated environments on coherence and clarity at 13,800x lower cost. Evaluated across 4 model families and 8 agent harness frameworks, we find that harness engineering boosts performance by up to 15.7 percentage points over a bare ReAct baseline, completion remains the primary axis of variation with no model saturating the benchmark, and automated generation enables evaluation at a scale previously infeasible. Beyond static benchmarking, ClawEnvKit enables live evaluation: users describe a desired capability in natural language and obtain a verified environment on demand, turning evaluation into a continuous, user-driven process. The same mechanism serves as an on-demand training environment generator, producing task distributions that adapt to an agent's current weaknesses rather than being bounded by existing user logs.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 7 minor

Summary. The manuscript proposes EVT (Euclidean Enhanced Vision Transformer), which extends the authors' prior RMT backbone in two ways: (1) replacing the Manhattan distance-based spatial decay matrix with a Euclidean distance-based one, and (2) replacing the horizontal/vertical decomposed attention with a 1D spatially-independent token grouping mechanism (EuSA_g / EuSA_d). The paper presents extensive experiments across ImageNet-1K classification, COCO object detection/instance segmentation, ADE20K semantic segmentation, robustness benchmarks, and throughput comparisons, along with detailed ablation studies. EVT-L reportedly achieves 86.6% top-1 accuracy on ImageNet-1K at 384×384 resolution. **Note: The abstract provided with this submission describes an entirely different paper ('ClawEnvKit: Automatic Environment Generation for Claw-Like Agents'). The full text is about EVT/Vision Transformers. This mismatch must be resolved; my review addresses the EVT paper as presented in the full text.**

Significance. The paper delivers a strong empirical contribution: the EVT family consistently outperforms RMT and other contemporary backbones across classification, detection, segmentation, and robustness (Tables 7–12), with favorable throughput (Table 12). The 1D grouping design is a clean, well-motivated simplification that reduces complexity from O(N^{1.5}) to O(Nk) while improving speed. The ablation studies (Tables 13–20) are thorough and systematically isolate each component's contribution. The roadmap tables (Tables 16, 18) transparently show the incremental gains from each modification.

major comments (3)
  1. Abstract/paper mismatch: The abstract describes 'ClawEnvKit,' a system for generating environments for claw-like agents, while the full text is entirely about EVT, a Vision Transformer backbone. This is a fundamental submission error that must be corrected before any publication decision can be finalized.
  2. §3.3, Table 1: The JS divergence argument used to motivate Euclidean over Manhattan distance is methodologically weak. The reference model ('EVT-T without any decay matrix') retains Conv Stem, CPE, 1D grouping, and LCE — all components of the EVT design. While the skeptic's note that these components do not inherently bias toward Euclidean over Manhattan is partially valid, the reference model is still an EVT variant, making the comparison somewhat self-referential. More importantly, the 0.3% accuracy gap between Manhattan (82.7%) and Euclidean (83.0%) in Table 1 is reported as a single-run result with no error bars, no multiple seeds, and no significance test. Given that this gap is a central design justification, at least 3-seed runs with standard deviations should be reported. The same concern applies to Table 16 (MaSA→EuSA: +0.4% acc, +1.2% mIoU).
  3. §3.3, points (2)–(9): The nine 'perspectives' arguing for Euclidean over Manhattan distance are largely qualitative assertions without formal proofs or targeted empirical verification. For example, point (5) claims eigenvector structure of the Laplacian 'captures isotropic spatial harmonics' without computing them; point (6) claims entropy maximization without proof. These are reasonable intuitions but do not constitute rigorous analysis. If these arguments are retained, they should be clearly labeled as informal motivation rather than analysis, or supported with concrete computations.
minor comments (7)
  1. Table 1: 'Mahattan' should be 'Manhattan' (typo).
  2. Table 2 caption: 'The JS divergence between different models' is vague; clarify what distributions are being compared (attention score distributions).
  3. §3.3, Eq. (4): The JS divergence formula uses D_KL(P||Q) but the standard definition uses D_KL(P||M) and D_KL(Q||M) where M = (P+Q)/2. The formula as written is correct but the notation could be clearer about which distribution is P (the reference model's attention) and which is Q (the decay matrix).
  4. Table 14: Several entries report 'Nan' for Minkowski p≥4 and RBF variants. It would be helpful to briefly note the training configuration (learning rate, warmup) used for these experiments, as NaN issues can sometimes be mitigated by hyperparameter adjustments.
  5. Figure 5: The loss curves comparing Euclidean and Manhattan distances lack axis labels and a clear legend. Please add labels for the x-axis (epochs), y-axis (loss), and identify which curve corresponds to which distance.
  6. §4.6, Table 16: The 'deeper' row increases parameters from 14M to 15M and FLOPs from 2.5G to 2.7G. Clarify what architectural change was made (additional layers? wider channels?).
  7. The paper would benefit from a brief discussion of limitations — e.g., the manually set decay coefficient γ (Eq. 35) and group size k are hyperparameters that may require task-specific tuning.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the thorough and constructive review. We address each major comment below.

read point-by-point responses

  1. Referee: Abstract/paper mismatch: The abstract describes 'ClawEnvKit' while the full text is about EVT.

    Authors: We sincerely apologize for this error. During the submission process, an incorrect abstract from a different manuscript was inadvertently included. The abstract in the full text (describing EVT) is the correct one. We will ensure the abstract is replaced with the proper EVT abstract in the revised manuscript. This is purely a clerical error and does not affect any of the technical content. revision: yes

  2. Referee: The JS divergence argument is methodologically weak: the reference model is self-referential (retains EVT components), and the 0.3% accuracy gap is a single-run result with no error bars or significance test. Same concern applies to Table 16.

    Authors: We thank the referee for this careful observation and acknowledge two valid concerns. (a) The reference model ('EVT-T without any decay matrix') does retain Conv Stem, CPE, 1D grouping, and LCE, making it an EVT variant rather than a vanilla ViT. We agree this makes the comparison somewhat self-referential. However, the purpose of this experiment is not to show that Euclidean decay is universally superior in isolation, but to measure which decay matrix distribution most closely matches the attention patterns that a model without explicit decay learns on its own. The reference model serves as a proxy for 'learned spatial attention patterns,' and the JS divergence measures which decay prior (Manhattan vs. Euclidean) better aligns with those learned patterns. We will revise the text to explicitly acknowledge that the reference model is an EVT variant and to clarify the intended interpretation. (b) Regarding the lack of multiple seeds: we agree that reporting single-run results for a 0.3% gap is insufficient. We will conduct 3-seed runs for the Manhattan vs. Euclidean comparison on EVT-T and report mean ± standard deviation in the revised manuscript. We note that the Euclidean advantage is also supported by broader experimental evidence across all model scales (Tables 7, 13, 16) and downstream tasks (Tables 8–11), where EVT consistently outperforms RMT (which uses Manhattan distance). We will add multi-seed results to Table 16 as well. revision: partial

  3. Referee: The nine 'perspectives' arguing for Euclidean over Manhattan are largely qualitative assertions without formal proofs or targeted empirical verification (e.g., point 5 on Laplacian eigenvectors, point 6 on entropy maximization).

    Authors: We appreciate this feedback. The referee is correct that points (2)–(9) in §3.3 are largely informal motivational arguments rather than rigorous analysis. We will revise the manuscript to clearly label these as 'informal motivation' rather than 'analysis.' However, we would note that several points do contain concrete mathematical content: point (4) provides explicit gradient formulas showing that the L2 gradient is smooth and direction-aware while the L1 gradient is piecewise constant and discontinuous; point (8) provides the second derivative showing the L2 optimization landscape is well-conditioned while L1 has zero second derivatives almost everywhere. These are formal observations, not merely intuitions. For point (5) (eigenvector structure of the Laplacian) and point (6) (entropy maximization), the referee is correct that we do not compute the eigenvectors or prove the entropy claim. We will either add concrete computations for small token grids to substantiate these claims, or remove or substantially soften these points if the computations do not support the claims as strongly as stated. We commit to verifying these claims empirically before retaining them. The core empirical justification for Euclidean over Manhattan distance rests on: (i) the JS divergence experiment (Table 1), (ii) the consistent accuracy improvements across all model scales and tasks (Tables 7–11, 16), (iii) the training stability comparison (Figure 5), and (iv) the ablation showing other distance functions (Minkowski, RBF) cause instability (Table 14). The nine perspectives are supplementary motivation, not the primary evidence, and we will make this clear in the revision. revision: partial

Circularity Check

1 steps flagged

Minor circularity in the JS divergence argument (§3.3, Tab. 1): the reference model is an EVT variant, so the 'ground truth' attention distribution is shaped by EVT's own architectural choices. The central empirical claims are independently grounded in external benchmarks.

specific steps
  1. self definitional [§3.3, Tab. 1 (JS divergence analysis)]
    "we train an EVT-T model without any decay matrix. Then, we analyze the correlation between its attention score distribution and those of different decay matrices. We use Jensen-Shannon (JS) divergence to measure the similarity... Based on the JS divergence values, the distribution of the Euclidean distance-based decay matrix exhibits higher similarity to the distribution of the standard attention scores in the trained model, which also leads to better performance."

    The reference distribution is obtained from 'an EVT-T model without any decay matrix.' This model retains EVT's other architectural choices (Conv Stem, CPE, 1D grouping, LCE). The paper then measures how closely the Euclidean decay matrix matches this reference and uses the lower JS divergence to justify the Euclidean choice. The reference distribution is thus shaped by EVT's own design decisions, making the comparison partially self-referential: the 'ground truth' attention pattern is produced by an EVT variant, and the decay matrix that best matches it is then selected for EVT. However, the circularity is indirect: 1D grouping flattens spatial dimensions before grouping, and CPE/Conv Stem provide positional information through convolutions, none of which inherently bias toward Euclidean-

full rationale

The central empirical claims of the paper (EVT outperforms RMT and other backbones on ImageNet, COCO, ADE20K, robustness benchmarks) are grounded in external benchmarks and are not circular. The JS divergence argument in §3.3 that motivates the Euclidean distance choice is mildly circular: it uses an EVT-T model without decay as the reference distribution, then measures how well different decay matrices match that distribution. Since the reference model shares EVT's architecture (Conv Stem, CPE, 1D grouping, LCE), the 'ground truth' attention pattern is shaped by the same design family. However, this is a supporting argument, not the load-bearing claim. The primary evidence for Euclidean over Manhattan is the empirical ablation (Tab. 1: 82.7% Manhattan vs 83.0% Euclidean; Tab. 16: +0.4% acc, +1.2% mIoU), which is independently measured on external benchmarks regardless of the JS divergence analysis. The nine qualitative perspectives in §3.3 (points 2–9) are standard mathematical arguments about gradient smoothness, spectral properties, and entropy that do not depend on self-citation. The paper cites RMT [10] as prior work by the same authors, but this is a legitimate precursor, not a self-citation used to forbid alternatives. Score 2: one minor self-referential step in a supporting argument, central claims independently grounded.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 0 invented entities

EVT does not invent new entities (no new particles, forces, or dimensions). The architectural components (Euclidean decay matrix, 1D grouping, EuSA_g, EuSA_d, LCE) are engineering constructs, not postulated physical or mathematical objects requiring independent evidence. The free parameters (γ, k, model dimensions) are standard hyperparameters in backbone design. The most notable axiom is the circular JS divergence reference, which is ad hoc to this paper.

free parameters (3)
  • γ (decay coefficient per head) = γ_n = 1 − 2^{−3−n} (Eq. 35)
    Manually set hyperparameter controlling receptive field per attention head. The specific functional form is not derived from first principles; ablation in Tab. 15 shows alternatives (1−2^{−7−n}) perform similarly, suggesting the exact form is not critical.
  • k (tokens per group) = 98 (Stages 1–3), 49 (Stage 4)
    Manually set group size that controls the accuracy-speed trade-off. Table 5 specifies k=98 for most stages and k=49 for Stage 4. Table 20 shows performance varies with k, but the choice is heuristic.
  • Model width/depth per variant = EVT-T: C=64, 2 blocks S1; EVT-L: C=128, 6 blocks S1, etc. (Tab. 5)
    Standard architecture scaling choices aligned with prior work conventions (Swin, RMT). Not independently derived.
axioms (4)
  • domain assumption Human visual attention decays radially with distance from the center of the visual field, and Euclidean distance models this better than Manhattan distance.
    Invoked in §3.3 and §1 to motivate the switch from Manhattan to Euclidean distance. Cited to [13] (Peripheral Vision Transformer). This is a domain assumption about biological vision transferred to neural network design.
  • ad hoc to paper The attention score distribution of a well-trained ViT without a decay matrix is a valid reference for evaluating which decay matrix is 'better.'
    Invoked in §3.3(1) to justify Euclidean over Manhattan via JS divergence. The reference model is EVT-T without decay, making this assumption circular with respect to the architecture being proposed.
  • domain assumption 1D token grouping with a 2D Euclidean decay matrix provides sufficient spatial priors to compensate for the loss of explicit 2D spatial structure in grouping.
    Invoked in §3.4 to justify replacing RMT's decomposed attention with 1D grouping. Supported empirically by Tab. 3 (1D+decay: 83.9% vs 2D+decay: 83.8%) but not derived theoretically.
  • standard math Standard self-attention can learn spatial priors but requires large-scale data and computation to do so effectively.
    Invoked in §3.3(3). This is a widely held view in the ViT literature, supported by the contrast between vanilla ViT and DINOv2.

pith-pipeline@v1.1.0-glm · 34343 in / 3149 out tokens · 192003 ms · 2026-07-05T11:26:42.060534+00:00 · methodology

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Overeager Coding Agents: Measuring Out-of-Scope Actions on Benign Tasks

    cs.SE 2026-05 conditional novelty 7.0

    The paper presents OverEager-Gen, a 500-scenario benchmark showing that removing consent declarations from prompts increases overeager actions by 11.9-17.2 percentage points across models, with agent framework choice ...

  2. ClawForge: Generating Executable Interactive Benchmarks for Command-Line Agents

    cs.AI 2026-05 conditional novelty 7.0

    ClawForge supplies a generator that turns scenario templates into reproducible command-line tasks testing state conflict handling, where the strongest frontier model scores only 45.3 percent strict accuracy.

  3. ClawForge: Generating Executable Interactive Benchmarks for Command-Line Agents

    cs.AI 2026-05 unverdicted novelty 7.0

    ClawForge is a generator framework that creates reproducible executable benchmarks for command-line agents under state conflict, with ClawForge-Bench showing frontier models reach at most 45.3% strict accuracy and tha...

Reference graph

Works this paper leans on

102 extracted references · 102 canonical work pages · cited by 2 Pith papers · 11 internal anchors

  1. [1]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy, L. Beyer, A. Kolesnikovet al., “An image is worth 16x16 words: Transformers for image recognition at scale,” in ICLR, 2021

    work page 2021

  2. [2]

    Uniformer: Unified transformer for efficient spatiotemporal rep- resentation learning,

    K. Li, Y. Wang, P . Gao, G. Song, Y. Liu, H. Li, and Y. Qiao, “Uniformer: Unified transformer for efficient spatiotemporal rep- resentation learning,” 2022

    work page 2022

  3. [3]

    Swin transformer: Hierarchical vision transformer using shifted windows,

    Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” inICCV, 2021

    work page 2021

  4. [4]

    Going deeper with image transformers,

    H. Touvron, M. Cord, A. Sablayrolles, G. Synnaeve, and H. J ´egou, “Going deeper with image transformers,” inICCV, 2021

    work page 2021

  5. [5]

    Rethinking Local Perception in Lightweight Vision Transformer

    Q. Fan, H. Huang, J. Guan, and R. He, “Rethinking lo- cal perception in lightweight vision transformer,”ArXiv, vol. abs/2303.17803, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  6. [6]

    CvT: Introducing Convolutions to Vision Transformers

    H. Wu, B. Xiao, N. Codella, M. Liu, X. Dai, L. Yuan, and L. Zhang, “Cvt: Introducing convolutions to vision transformers,”arXiv preprint arXiv:2103.15808, 2021

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  7. [7]

    Cmt: Convolutional neural networks meet vision transformers,

    J. Guo, K. Han, H. Wu, C. Xu, Y. Tang, C. Xu, and Y. Wang, “Cmt: Convolutional neural networks meet vision transformers,” inCVPR, 2022

    work page 2022

  8. [8]

    Lite vision transformer with enhanced self-attention,

    C. Yang, Y. Wang, J. Zhanget al., “Lite vision transformer with enhanced self-attention,” inCVPR, 2022

    work page 2022

  9. [9]

    Neighborhood attention transformer,

    A. Hassani, S. Walton, J. Li, S. Li, and H. Shi, “Neighborhood attention transformer,” inCVPR, 2023

    work page 2023

  10. [10]

    Rmt: Retentive networks meet vision transformers,

    Q. Fan, H. Huang, M. Chen, H. Liu, and R. He, “Rmt: Retentive networks meet vision transformers,” inCVPR, 2024

    work page 2024

  11. [11]

    Retentive Network: A Successor to Transformer for Large Language Models

    Y. Sun, L. Dong, S. Huang, S. Ma, Y. Xia, J. Xue, J. Wang, and F. Wei, “Retentive network: A successor to Transformer for large language models,”ArXiv, vol. abs/2307.08621, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  12. [12]

    Train short, test long: At- JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 17 tention with linear biases enables input length extrapolation,

    O. Press, N. Smith, and M. Lewis, “Train short, test long: At- JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 17 tention with linear biases enables input length extrapolation,” in ICLR, 2022

    work page 2021

  13. [13]

    Peripheral vision transformer,

    J. Min, Y. Zhao, C. Luoet al., “Peripheral vision transformer,” in NeurIPS, 2022

    work page 2022

  14. [14]

    Cswin transformer: A general vision transformer backbone with cross-shaped windows,

    X. Dong, J. Bao, D. Chenet al., “Cswin transformer: A general vision transformer backbone with cross-shaped windows,” in CVPR, 2022

    work page 2022

  15. [15]

    Vision trans- former with deformable attention,

    Z. Xia, X. Pan, S. Song, L. E. Li, and G. Huang, “Vision trans- former with deformable attention,” inCVPR, 2022

    work page 2022

  16. [16]

    Davit: Dual attention vision transformers,

    M. Ding, B. Xiao, N. Codellaet al., “Davit: Dual attention vision transformers,” inECCV, 2022

    work page 2022

  17. [17]

    Biformer: Vision transformer with bi-level routing attention,

    L. Zhu, X. Wang, Z. Ke, W. Zhang, and R. Lau, “Biformer: Vision transformer with bi-level routing attention,” inCVPR, 2023

    work page 2023

  18. [18]

    A convnet for the 2020s,

    Z. Liu, H. Mao, C.-Y. Wuet al., “A convnet for the 2020s,” in CVPR, 2022

    work page 2022

  19. [19]

    Semantic equitable clustering: A simple, fast and effective strategy for vision trans- former,

    Q. Fan, H. Huang, M. Chen, and R. He, “Semantic equitable clustering: A simple, fast and effective strategy for vision trans- former,” inICCV, 2025

    work page 2025

  20. [20]

    Vision transformer with sparse scan prior,

    ——, “Vision transformer with sparse scan prior,” 2024

    work page 2024

  21. [21]

    Twins: Revisiting the design of spatial attention in vision transformers,

    X. Chu, Z. Tian, Y. Wang, B. Zhang, H. Ren, X. Wei, H. Xia, and C. Shen, “Twins: Revisiting the design of spatial attention in vision transformers,” inNeurIPS, 2021

    work page 2021

  22. [22]

    Pyramid vision transformer: A versatile backbone for dense prediction without convolutions,

    W. Wang, E. Xie, X. Li, D.-P . Fan, K. Song, D. Liang, T. Lu, P . Luo, and L. Shao, “Pyramid vision transformer: A versatile backbone for dense prediction without convolutions,” inICCV, 2021

    work page 2021

  23. [23]

    Pvtv2: Improved baselines with pyramid vision trans- former,

    ——, “Pvtv2: Improved baselines with pyramid vision trans- former,”Computational Visual Media, vol. 8, no. 3, pp. 1–10, 2022

    work page 2022

  24. [24]

    Lightweight vision transformer with bidirectional interaction,

    Q. Fan, H. Huang, X. Zhou, and R. He, “Lightweight vision transformer with bidirectional interaction,” inNeurIPS, 2023

    work page 2023

  25. [25]

    Multi-scale vision longformer: A new vision transformer for high-resolution image encoding,

    P . Zhang, X. Dai, J. Yang, B. Xiao, L. Yuan, L. Zhang, and J. Gao, “Multi-scale vision longformer: A new vision transformer for high-resolution image encoding,” inICCV, 2021

    work page 2021

  26. [26]

    RegionViT: Regional-to- Local Attention for Vision Transformers,

    C.-F. R. Chen, R. Panda, and Q. Fan, “RegionViT: Regional-to- Local Attention for Vision Transformers,” inICLR, 2022

    work page 2022

  27. [27]

    Visual Parser: Representing Part-whole Hierarchies with Transformers

    S. Sun, X. Yue, S. Bai, and P . Torr, “Visual parser: Represent- ing part-whole hierarchies with transformers,”arXiv preprint arXiv:2107.05790, 2021

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  28. [28]

    Perceiver: General perception with iterative atten- tion,

    A. Jaegle, F. Gimeno, A. Brock, A. Zisserman, O. Vinyals, and J. Carreira, “Perceiver: General perception with iterative atten- tion,” inICML, 2021

    work page 2021

  29. [29]

    Msg- transformer: Exchanging local spatial information by manipulat- ing messenger tokens,

    J. Fang, L. Xie, X. Wang, X. Zhang, W. Liu, and Q. Tian, “Msg- transformer: Exchanging local spatial information by manipulat- ing messenger tokens,” inCVPR, 2022

    work page 2022

  30. [30]

    Hrformer: High-resolution transformer for dense prediction,

    Y. Yuan, R. Fu, L. Huang, W. Lin, C. Zhang, X. Chen, and J. Wang, “Hrformer: High-resolution transformer for dense prediction,” in NeurIPS, 2021

    work page 2021

  31. [31]

    Multi-scale high-resolution vision transformer for semantic segmentation,

    J. Gu, H. Kwon, D. Wang, W. Ye, M. Li, Y.-H. Chen, L. Lai, V . Chandra, and D. Z. Pan, “Multi-scale high-resolution vision transformer for semantic segmentation,” inCVPR, 2022

    work page 2022

  32. [32]

    When Do We Not Need Larger Vision Models?

    B. Shi, Z. Wu, M. Mao, X. Wang, and T. Darrell, “When do we not need larger vision models?”arXiv preprint arXiv:2403.13043, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  33. [33]

    Hiri-vit: Scaling vision trans- former with high resolution inputs,

    T. Yao, Y. Li, Y. Pan, and T. Mei, “Hiri-vit: Scaling vision trans- former with high resolution inputs,”TP AMI, 2024

    work page 2024

  34. [34]

    Swin transformer v2: Scaling up capacity and resolution,

    Z. Liu, H. Hu, Y. Lin, Z. Yao, Z. Xie, Y. Wei, J. Ning, Y. Cao, Z. Zhang, L. Dong, F. Wei, and B. Guo, “Swin transformer v2: Scaling up capacity and resolution,” inCVPR, 2022

    work page 2022

  35. [35]

    Internimage: Exploring large-scale vision foundation models with deformable convolutions,

    W. Wang, J. Dai, Z. Chen, Z. Huang, Z. Li, X. Zhu, X. Hu, T. Lu, L. Lu, H. Liet al., “Internimage: Exploring large-scale vision foundation models with deformable convolutions,” inCVPR, 2023

    work page 2023

  36. [36]

    Not all patches are what you need: Expediting vision transformers via token reorganizations,

    Y. Liang, C. Ge, Z. Tong, Y. Song, J. Wang, and P . Xie, “Not all patches are what you need: Expediting vision transformers via token reorganizations,” inICLR, 2022

    work page 2022

  37. [37]

    Dynam- icvit: Efficient vision transformers with dynamic token sparsifi- cation,

    Y. Rao, W. Zhao, B. Liu, J. Lu, J. Zhou, and C.-J. Hsieh, “Dynam- icvit: Efficient vision transformers with dynamic token sparsifi- cation,” inNeurIPS, 2021

    work page 2021

  38. [38]

    Token merging: Your ViT but faster,

    D. Bolya, C.-Y. Fu, X. Dai, P . Zhang, C. Feichtenhofer, and J. Hoffman, “Token merging: Your ViT but faster,” inICLR, 2023

    work page 2023

  39. [39]

    Vision transformer with super token sampling,

    H. Huang, X. Zhou, J. Cao, R. He, and T. Tan, “Vision transformer with super token sampling,” inCVPR, 2023

    work page 2023

  40. [40]

    Conformer: Local features coupling global representations for visual recognition,

    Z. Peng, W. Huang, S. Gu, L. Xie, Y. Wang, J. Jiao, and Q. Ye, “Conformer: Local features coupling global representations for visual recognition,” inICCV, 2021

    work page 2021

  41. [41]

    Mixformer: Mixing features across windows and dimensions,

    Q. Chen, Q. Wu, J. Wang, Q. Hu, T. Hu, E. Ding, J. Cheng, and J. Wang, “Mixformer: Mixing features across windows and dimensions,” inCVPR, 2022

    work page 2022

  42. [42]

    Inception transformer,

    C. Si, W. Yu, P . Zhou, Y. Zhou, X. Wang, and S. YAN, “Inception transformer,” inNeurIPS, 2022

    work page 2022

  43. [43]

    Fast vision transformers with hilo attention,

    Z. Pan, J. Cai, and B. Zhuang, “Fast vision transformers with hilo attention,” inNeurIPS, 2022

    work page 2022

  44. [44]

    CoAtNet: Marrying Convolution and Attention for All Data Sizes

    Z. Dai, H. Liu, Q. V . Le, and M. Tan, “Coatnet: Marrying convolution and attention for all data sizes,”arXiv preprint arXiv:2106.04803, 2021

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  45. [45]

    Orthogonal transformer: An effi- cient vision transformer backbone with token orthogonalization,

    H. Huang, X. Zhou, and R. He, “Orthogonal transformer: An effi- cient vision transformer backbone with token orthogonalization,” inNeurIPS, 2022

    work page 2022

  46. [46]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmaret al., “Attention is all you need,” inNeurIPS, 2017

    work page 2017

  47. [47]

    Conditional positional encodings for vision transformers,

    X. Chu, Z. Tian, B. Zhang, X. Wang, and C. Shen, “Conditional positional encodings for vision transformers,” inICLR, 2023

    work page 2023

  48. [48]

    EVA-02: A Visual Representation for Neon Genesis

    Y. Fang, Q. Sun, X. Wang, T. Huang, X. Wang, and Y. Cao, “Eva-02: A visual representation for neon genesis,”arXiv preprint arXiv:2303.11331, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  49. [49]

    Flatten trans- former: Vision transformer using focused linear attention,

    D. Han, X. Pan, Y. Han, S. Song, and G. Huang, “Flatten trans- former: Vision transformer using focused linear attention,” in ICCV, 2023

    work page 2023

  50. [50]

    Agent attention: On the integration of softmax and linear attention,

    D. Han, T. Ye, Y. Han, Z. Xia, S. Song, and G. Huang, “Agent attention: On the integration of softmax and linear attention,” in ECCV, 2024

    work page 2024

  51. [51]

    Roformer: Enhanced transformer with rotary position embedding,

    J. Su, Y. Lu, S. Pan, B. Wen, and Y. Liu, “Roformer: Enhanced transformer with rotary position embedding,” 2021

    work page 2021

  52. [52]

    Maxvit: Multi-axis vision transformer,

    Z. Tu, H. Talebi, H. Zhang, F. Yang, P . Milanfar, A. Bovik, and Y. Li, “Maxvit: Multi-axis vision transformer,” inECCV, 2022

    work page 2022

  53. [53]

    Multi-scale vmamba: Hierarchy in hierarchy visual state space model,

    Y. Shi, M. Dong, and C. Xu, “Multi-scale vmamba: Hierarchy in hierarchy visual state space model,” inNeurIPS, 2024

    work page 2024

  54. [54]

    Moat: Alternating mobile convolu- tion and attention brings strong vision models,

    C. Yang, S. Qiao, Q. Yuet al., “Moat: Alternating mobile convolu- tion and attention brings strong vision models,” inICLR, 2023

    work page 2023

  55. [55]

    Visual Attention Network

    M.-H. Guo, C.-Z. Lu, Z.-N. Liu, M.-M. Cheng, and S.-M. Hu, “Visual attention network,”arXiv preprint arXiv:2202.09741, 2022

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  56. [56]

    Conv2Former: A Simple Transformer-Style ConvNet for Visual Recognition

    Q. Hou, C.-Z. Lu, M.-M. Cheng, and J. Feng, “Conv2former: A simple transformer-style convnet for visual recognition,”arXiv preprint arXiv:2211.11943, 2022

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  57. [57]

    Scaling up your kernels: Large kernel design in convnets towards universal representations,

    Y. Zhang, X. Ding, and X. Yue, “Scaling up your kernels: Large kernel design in convnets towards universal representations,” TP AMI, 2025

    work page 2025

  58. [58]

    Learned queries for efficient local attention,

    M. Arar, A. Shamir, and A. H. Bermano, “Learned queries for efficient local attention,” inCVPR, 2022

    work page 2022

  59. [59]

    Global context vision transformers,

    A. Hatamizadeh, H. Yin, G. Heinrich, J. Kautz, and P . Molchanov, “Global context vision transformers,” inICML, 2023

    work page 2023

  60. [60]

    Scale-aware modulation meet transformer,

    W. Lin, Z. Wu, J. Chen, J. Huang, and L. Jin, “Scale-aware modulation meet transformer,” inICCV, 2023

    work page 2023

  61. [61]

    Transnext: Robust foveal visual perception for vision transformers,

    D. Shi, “Transnext: Robust foveal visual perception for vision transformers,” inCVPR, 2024

    work page 2024

  62. [62]

    Overlock: An overview-first-look-closely-next convnet with context-mixing dynamic kernels,

    M. Lou and Y. Yu, “Overlock: An overview-first-look-closely-next convnet with context-mixing dynamic kernels,” inCVPR, 2025

    work page 2025

  63. [63]

    Training data-efficient image transformers & distillation through attention,

    H. Touvron, M. Cord, M. Douzeet al., “Training data-efficient image transformers & distillation through attention,” inICML, 2021

    work page 2021

  64. [64]

    CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification,

    C.-F. R. Chen, Q. Fan, and R. Panda, “CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification,” in ICCV, 2021

    work page 2021

  65. [65]

    Focal self-attention for local-global interactions in vision transformers,

    J. Yang, C. Li, P . Zhang, X. Dai, B. Xiao, L. Yuan, and J. Gao, “Focal self-attention for local-global interactions in vision transformers,” inNeurIPS, 2021

    work page 2021

  66. [66]

    Sucheng ren, xingyi yang, songhua liu, xinchao wang,

    S.-F. S. guided Transformer with Evolving Token Reallocation, “Sucheng ren, xingyi yang, songhua liu, xinchao wang,” inICCV, 2023

    work page 2023

  67. [67]

    Demystify mamba in vision: A linear attention perspective,

    D. Han, Z. Wang, Z. Xia, Y. Han, Y. Pu, C. Ge, J. Song, S. Song, B. Zheng, and G. Huang, “Demystify mamba in vision: A linear attention perspective,” inNeurIPS, 2024

    work page 2024

  68. [68]

    Vssd: Vision mamba with non-causal state space duality,

    Y. Shi, M. Dong, M. Li, and C. Xu, “Vssd: Vision mamba with non-causal state space duality,” inICCV, 2025

    work page 2025

  69. [69]

    Quadtree attention for vision transformers,

    S. Tang, J. Zhang, S. Zhuet al., “Quadtree attention for vision transformers,” inICLR, 2022

    work page 2022

  70. [70]

    Scalablevit: Rethinking the context-oriented generalization of vision transformer,

    R. Yang, H. Ma, J. Wu, Y. Tang, X. Xiao, M. Zheng, and X. Li, “Scalablevit: Rethinking the context-oriented generalization of vision transformer,” inECCV, 2022

    work page 2022

  71. [71]

    Crossformer++: A versatile vision transformer hinging on cross-scale attention,

    W. Wang, W. Chen, Q. Qiu, L. Chen, B. Wu, B. Lin, X. He, and W. Liu, “Crossformer++: A versatile vision transformer hinging on cross-scale attention,”TP AMI, 2023

    work page 2023

  72. [72]

    Contextual transformer net- works for visual recognition,

    Y. Li, T. Yao, Y. Pan, and T. Mei, “Contextual transformer net- works for visual recognition,”TP AMI, 2022. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 18

    work page 2022

  73. [73]

    ViTAEv2: Vision Transformer Advanced by Exploring Inductive Bias for Image Recognition and Beyond

    Q. Zhang, Y. Xu, J. Zhang, and D. Tao, “Vitaev2: Vision trans- former advanced by exploring inductive bias for image recogni- tion and beyond,”arXiv preprint arXiv:2202.10108, 2022

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  74. [74]

    Mvitv2: Improved multiscale vision trans- formers for classification and detection,

    Y. Li, C.-Y. Wu, H. Fan, K. Mangalam, B. Xiong, J. Malik, and C. Feichtenhofer, “Mvitv2: Improved multiscale vision trans- formers for classification and detection,” inCVPR, 2022

    work page 2022

  75. [75]

    Crossformer: A versatile vision transformer hinging on cross- scale attention,

    W. Wang, L. Yao, L. Chen, B. Lin, D. Cai, X. He, and W. Liu, “Crossformer: A versatile vision transformer hinging on cross- scale attention,” inICLR, 2022

    work page 2022

  76. [76]

    Mpvit: Multi-path vision transformer for dense prediction,

    Y. Lee, J. Kim, J. Willette, and S. J. Hwang, “Mpvit: Multi-path vision transformer for dense prediction,” inCVPR, 2022

    work page 2022

  77. [77]

    Vision transformer adapter for dense predictions,

    Z. Chen, Y. Duan, W. Wang, J. He, T. Lu, J. Dai, and Y. Qiao, “Vision transformer adapter for dense predictions,” inICLR, 2023

    work page 2023

  78. [78]

    Do imagenet classifiers generalize to imagenet?

    B. Recht, R. Roelofs, L. Schmidt, and V . Shankar, “Do imagenet classifiers generalize to imagenet?” 2019

    work page 2019

  79. [79]

    Natural adversarial examples,

    D. Hendrycks, K. Zhao, S. Basart, J. Steinhardt, and D. Song, “Natural adversarial examples,” inCVPR, 2021

    work page 2021

  80. [80]

    The many faces of robustness: A critical analysis of out-of-distribution generalization,

    D. Hendrycks, S. Basart, N. Mu, S. Kadavath, F. Wang, E. Dorundo, R. Desai, T. Zhu, S. Parajuli, M. Guo, D. Song, J. Steinhardt, and J. Gilmer, “The many faces of robustness: A critical analysis of out-of-distribution generalization,” inICCV, 2021

    work page 2021

Showing first 80 references.