Pith. sign in

REVIEW 4 major objections 6 minor 38 references

Token Compression Meets Compact Vision Transformers: A Survey and Comparative Evaluation for Edge AI

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Token compression methods that accelerate large vision transformers fail when plugged into compact ones, collapsing AutoFormer-S accuracy from 81.66% to about 30% on ImageNet-1K until retraining restores it.

desk verdict A useful survey wrapped around a benchmark that looks like an integration artifact; the central negative result is not credible as presented. read the letter →

arxiv 2507.09702 v1 pith:LSEUMA7H submitted 2025-07-13 cs.CV

Phat Nguyen , Ngai-Man Cheung This is my paper · ORCID
classification cs.CV
keywords tokencompressionvisiontransformerscompactedgeAIpruningmergingmodelaccelerationImageNet-1K
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that token compression methods developed for large vision transformers do not transfer to structurally compact transformers used on edge devices. In the authors' experiments on AutoFormer-S, three plug-in methods—ToMe, PiToMe, and ToFu—collapse ImageNet-1K top-1 accuracy from 81.66% to roughly 30% even when only a few tokens are removed. Retraining the full network recovers accuracy to 78–79%, which the authors read as evidence that the compressed token embeddings are misaligned with the pretrained weights rather than that the information is destroyed. The paper also contributes a taxonomy that separates pruning, merging, and hybrid compression, and a deployment-oriented split between plug-in and trainable methods.

What carries the argument

The device that carries the argument is a two-part experimental protocol: a taxonomy that assigns each method to pruning, merging, or hybrid compression and to plug-in versus trainable deployment, and a benchmark that measures top-1 accuracy, GFLOPs, and throughput when those methods are applied to the AutoFormer-S compact backbone. The named mechanism proposed to explain the results is token-representation misalignment: plug-in compression alters the set of token embeddings, and the pretrained compact weights cannot consume the altered sequence, whereas full retraining realigns the weights to the compressed tokens and restores accuracy. This mechanism is what turns the accuracy collapse from a mere observation into a general claim about compact transformers.

What would settle it

Run the same three plug-in methods on a standard ViT-B with the same compression ratios and protocol used in Table III; if ViT-B stays near its baseline while AutoFormer-S collapses, the compact-backbone explanation is supported, whereas a similar collapse would point to an integration error in the paper's setup.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that token compression is not a one-size-fits-all solution when applied to compact backbones without adaptation. In the off-the-shelf setting, ToMe, PiToMe, and ToFu reduce AutoFormer-S accuracy from 81.66% to around 29–31% on ImageNet-1K, and the ablation in Table III shows the damage appears even at mild reductions of three to six tokens. After full retraining the same methods recover to 78.17–79.22%, while trainable methods transfer unevenly: DiffRate preserves 77.47% accuracy with a 1.5× throughput gain, Cropr drops below 70%, and DTEM fails to converge. The authors conclude that alignment between compressed token representations and pretrained network parameters is the critical factor, and that architecture-aware adaptation is needed before token optimization can be combined with structural compression for edge deployment.

Load-bearing premise

The paper's main empirical conclusion rests on the assumption that the off-the-shelf implementations of ToMe, PiToMe, and ToFu were integrated into AutoFormer-S correctly; if the integration was faulty, the observed collapse to roughly 30% accuracy would be an artifact rather than a property of compact transformers.

Editorial extensions

If this is right

  • Plug-in token compression methods cannot be treated as drop-in accelerators for structurally compressed transformers; they need adaptation or retraining.
  • Retraining a compact model with token compression restores most of the lost accuracy, so the bottleneck is alignment between token representations and network parameters, not information loss.
  • Token compression still delivers meaningful speedups on compact backbones, so it remains a complementary optimization to structural compression.
  • Trainable methods tuned on standard ViTs transfer unevenly: DiffRate holds 77%+ accuracy, DTEM fails to converge, and Cropr underperforms.
  • The findings motivate future work that jointly optimizes architecture and token compression rather than applying the two separately.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The absence of a standard-ViT control arm means the cleanest test of the paper's generalization claim is to run the same off-the-shelf plug-ins on a standard ViT-B with identical compression ratios; near-lossless results there would confirm compact backbones are the cause.
  • The sharp drop after removing only three to six tokens hints that the attention-based importance scores used by plug-in methods are poorly calibrated on NAS-searched compact models, which could be checked by comparing score distributions against standard ViTs.
  • If alignment is the bottleneck, a natural next step is to co-search architecture and compression during NAS so the compact model is born robust to token reduction, rather than compressing a fixed pretrained compact net.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. This paper combines a survey of token compression methods for Vision Transformers with an empirical evaluation of several such methods applied to the compact backbone AutoFormer-S on ImageNet-1K. The authors propose a taxonomy (pruning, merging, hybrid) and report two experiment settings: off-the-shelf plug-in compression (ToMe, PiToMe, ToFu) and retrained compression (Cropr, DiffRate, DTEM). The central empirical claim is that off-the-shelf token compression collapses AutoFormer-S accuracy from 81.66% to roughly 30%, while retraining recovers most of the loss, leading the authors to conclude that token compression is not a one-size-fits-all solution for compact backbones and must be adapted to them.

Significance. If the reported results were reliable, the paper would document an important negative result: off-the-shelf token compression methods, which are nearly lossless on standard ViTs, would fail catastrophically on structurally compact transformers. Such a finding would be highly relevant to edge-AI deployment and would motivate architecture-aware token compression. The survey portion also offers a useful taxonomy, particularly the hybrid pruning-plus-merging category, and the paper is explicit about deployment requirements in Table I. However, the empirical foundation of the central claim is not credible as presented: it lacks a standard-ViT control under the same protocol, shows an implausible 50-point drop inconsistent with published behavior of the same methods, exhibits an internally suspicious pattern across methods, and provides no code, seeds, or error bars. The paper's significance hinges on this experimental result, which currently appears to be an artifact rather than a genuine property of compact transformers.

major comments (4)
  1. [Section IV.B, Tables II and III] The central claim that off-the-shelf ToMe, PiToMe, and ToFu collapse AutoFormer-S accuracy to about 30% is not supported because the paper provides no standard ViT baseline run under the identical protocol. Published ToMe on ViT-B is nearly lossless at comparable per-layer token reductions, so the observed roughly 50-point drop is unexplained. Without a control experiment on a standard ViT, the reader cannot distinguish an integration bug (e.g., incorrectly handling the class token, applying compression at the wrong layer, or misinterpreting the keep-ratio parameter) from a real incompatibility with compact backbones.
  2. [Table III] The ablation in Table III shows a pattern that is more consistent with a shared implementation artifact than with a property of compact transformers: ToFu's accuracy increases as more tokens are pruned (30.72 at 3 pruned tokens vs. 31.99 at 18 pruned tokens), while ToMe's accuracy drops monotonically from 30.60 to 22.03. All three methods cluster near 30% even for the mildest compression of only 3 tokens per layer. This behavior is not explained by the paper, and no standard deviations, seeds, or multiple runs are reported, so the reader cannot assess whether the differences are meaningful.
  3. [Section IV.B, Table II] The retraining results weaken the paper's central narrative. After full fine-tuning, ToMe, PiToMe, and ToFu recover to 79.22, 78.74, and 78.17 accuracy, within 2.5 to 3.5 points of the uncompressed baseline. If the off-the-shelf collapse were caused by the compact backbone inherently discarding critical tokens, one would not expect retraining to recover so completely. The paper does not reconcile this, instead attributing the collapse to a 'mismatch' between token embeddings and pretrained weights without direct evidence. The retrained rows in fact suggest that the off-the-shelf integration, not the compact architecture, is the likely source of the reported catastrophe.
  4. [Section IV.A] The experimental setup is underspecified in ways that are load-bearing for reproducibility and for the off-the-shelf claim. The paper states that official implementations were followed, but it does not report which layers received compression, how many tokens are merged/pruned per layer, how the class token is handled, or the exact retraining recipe (epochs, learning rate, augmentation) for the 're-train' rows. Since the central result is a surprising negative outcome, this level of detail is insufficient; the authors should also release code to allow verification that the plug-in integration is correct.
minor comments (6)
  1. [Section IV.B] The text says 'without carefull adaptation'; this should be 'careful'.
  2. [Table II caption and Section IV.B] The term 'GLOPs' appears where 'GFLOPs' is intended in Table II and in the sentence about 'relative GLOPs improvement.'
  3. [Table III] The header says '#pruned tokens' but the text refers to 'compression ratios of 3 or 6 tokens.' Clarify whether these are tokens removed per transformer layer or total across the network, and specify the number of compression stages used.
  4. [Section III.A.2 and References] SPViT is cited as [37] in Section III.A.2 but as [5] in Section II.B and Reference [5] in the reference list; the numbering should be consistent.
  5. [Table I] The 'Hybrid' row for ToFu and DiffRate shows a dash for 'Compression Type' even though the text explains that hybrid combines pruning and merging; the table should state this explicitly for clarity.
  6. [Section IV.B] The reported throughput figures (img/s) are given without any mention of measurement warm-up, GPU power state, or repeated trials, which makes the speedup claims difficult to compare with other studies.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central claim is an empirical observation from external benchmarks, and the few self-citations are background references that do not support the result.

full rationale

The paper's central claim—that token compression methods underperform when directly applied to compact backbones without adaptation—is a direct empirical observation from Tables II and III, not a derived quantity. No parameter is fitted to a subset of data and then presented as a prediction; the off-the-shelf accuracies are measured by applying published plug-in methods to AutoFormer-S, and the retrained rows are separate experiments. The taxonomy section categorizes existing methods but does not rename a known result as a new prediction. The only self-citations, such as [24] and [31] from the Cheung group, appear in general background passages about edge AI and knowledge distillation and are not load-bearing for the empirical conclusion. The skeptical concern that the off-the-shelf integration of ToMe, PiToMe, and ToFu may be incorrect or unverified is a threat to experimental validity and reproducibility, not a form of circularity, because a buggy integration would falsify the paper's claim rather than make it true by definition. Therefore no circular step is identified and the score is 0.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central empirical claim depends on the choice of a single compact backbone (AutoFormer-S), on hand-selected compression ratios, and on unverified integration of official implementations into AutoFormer-S. No invented entities are introduced. The key axioms are domain assumptions about representativeness and correctness of the experimental protocol.

free parameters (3)
  • Number of merged/pruned tokens per stage = 9 in main results; 3 to 18 in ablation
    Chosen by hand. The main comparison uses a single compression ratio and the ablation spans 3-18 tokens, but no sensitivity analysis is reported for the retrained methods.
  • AutoFormer-S subnet selection = unspecified subnet from AutoFormer supernet
    A single compact backbone represents all 'compact designs'. No ablation over other compact architectures or subnet settings is provided.
  • Retraining hyperparameters = not reported (official defaults claimed)
    The retraining results depend on unreported training schedules, learning rates, and epochs, which are essential to interpret the recovery of ToMe, PiToMe, and ToFu.
assumptions (3)
  • domain assumption ImageNet-1K pretrained AutoFormer-S is a representative compact Vision Transformer
    The paper generalizes from one backbone to all compact transformers, stated in Section IV-A as 'representative compact transformer' without justification.
  • ad hoc to paper Official implementations of ToMe, PiToMe, ToFu, Cropr, DiffRate, DTEM are correctly integrated with AutoFormer-S
    The OTS results drop to ~30% accuracy after removing only 9 tokens, which is inconsistent with published ToMe behavior on standard ViTs. The paper does not verify the integration.
  • domain assumption Throughput and GFLOPs measurements are representative
    Single GPU, batch size 128, no measurement details such as warm-up, repeats, or variance. See Section IV-A.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Token Compression Meets Compact Vision Transformers: A Survey and Comparative Evaluation for Edge AI." pith.science (2026). https://pith.science/paper/LSEUMA7H

@misc{pith2026250709702,
  author       = {Pith},
  title        = {Pith review of: Token Compression Meets Compact Vision Transformers: A Survey and Comparative Evaluation for Edge AI},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LSEUMA7H}},
  note         = {Machine review of arXiv:2507.09702}
}
read the original abstract

Token compression techniques have recently emerged as powerful tools for accelerating Vision Transformer (ViT) inference in computer vision. Due to the quadratic computational complexity with respect to the token sequence length, these methods aim to remove less informative tokens before the attention layers to improve inference throughput. While numerous studies have explored various accuracy-efficiency trade-offs on large-scale ViTs, two critical gaps remain. First, there is a lack of unified survey that systematically categorizes and compares token compression approaches based on their core strategies (e.g., pruning, merging, or hybrid) and deployment settings (e.g., fine-tuning vs. plug-in). Second, most benchmarks are limited to standard ViT models (e.g., ViT-B, ViT-L), leaving open the question of whether such methods remain effective when applied to structurally compressed transformers, which are increasingly deployed on resource-constrained edge devices. To address these gaps, we present the first systematic taxonomy and comparative study of token compression methods, and we evaluate representative techniques on both standard and compact ViT architectures. Our experiments reveal that while token compression methods are effective for general-purpose ViTs, they often underperform when directly applied to compact designs. These findings not only provide practical insights but also pave the way for future research on adapting token optimization techniques to compact transformer-based networks for edge AI and AI agent applications.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

38 extracted references · 17 canonical work pages

  1. [36]

    Which tokens to use? investigating token reduction in vision transformers,

    J. B. Haurum, S. Escalera, G. W. Taylor, and T. B. Moeslund, “Which tokens to use? investigating token reduction in vision transformers,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 773–783

    work page 2023

  2. [1]

    EVit: Expediting vision transformers via token reorganizations,

    Y . Liang, C. GE, Z. Tong, Y . Song, J. Wang, and P. Xie, “EVit: Expediting vision transformers via token reorganizations,” in International Conference on Learn- ing Representations , 2022. [Online]. Available: https : //openreview.net/forum?id=BjyvwnXXVn

    2022

  3. [2]

    Dynamicvit: Efficient vision transformers with dynamic token sparsification,

    Y . Rao, W. Zhao, B. Liu, J. Lu, J. Zhou, and C.-J. Hsieh, “Dynamicvit: Efficient vision transformers with dynamic token sparsification,” in Advances in Neural Information Processing Systems , A. Beygelzimer, Y . Dauphin, P. Liang, and J. W. Vaughan, Eds., 2021. [Online]. Available: https://openreview.net/forum?id= jB0Nlbwlybm

    2021

  4. [3]

    Token cropr: Faster vits for quite a few tasks,

    B. Bergner, C. Lippert, and A. Mahendran, “Token cropr: Faster vits for quite a few tasks,” in Proceed- ings of the Computer Vision and Pattern Recognition Conference, 2025, pp. 9740–9750

    work page 2025

  5. [4]

    Adaptive token sampling for efficient vision trans- formers,

    M. Fayyaz, S. A. Koohpayegani, F. R. Jafari, et al. , “Adaptive token sampling for efficient vision trans- formers,” in European Conference on Computer Vision , Springer, 2022, pp. 396–414

    work page 2022

  6. [5]

    Spvit: Enabling faster vision transformers via latency-aware soft token pruning,

    Z. Kong, P. Dong, X. Ma, et al. , “Spvit: Enabling faster vision transformers via latency-aware soft token pruning,” in Proceedings of the European Conference on Computer Vision (ECCV) , 2022

    work page 2022

  7. [6]

    Token merging: Your ViT but faster,

    D. Bolya, C.-Y . Fu, X. Dai, P. Zhang, C. Feichtenhofer, and J. Hoffman, “Token merging: Your ViT but faster,” in International Conference on Learning Representa- tions, 2023

    2023

  8. [8]

    Not all tokens are equal: Human-centric visual analysis via token cluster- ing transformer,

    W. Zeng, S. Jin, W. Liu, et al. , “Not all tokens are equal: Human-centric visual analysis via token cluster- ing transformer,” in Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition , 2022, pp. 11 101–11 111

    2022

Show all 38 references
  1. [9]

    Ac- celerating transformers with spectrum-preserving token merging,

    C. Tran, D. MH Nguyen, M.-D. Nguyen, et al. , “Ac- celerating transformers with spectrum-preserving token merging,” Advances in Neural Information Processing Systems, vol. 37, pp. 30 772–30 810, 2024

    2024

  2. [10]

    Self-slimmed vision transformer,

    Z. Zong, K. Li, G. Song, et al. , “Self-slimmed vision transformer,” in Proceedings of the European Confer- ence on Computer Vision (ECCV) , 2022

    2022

  3. [11]

    Multi-scale hybrid vision transformer and sinkhorn tokenizer for sewer defect classification,

    J. B. Haurum, M. Madadi, S. Escalera, and T. B. Moeslund, “Multi-scale hybrid vision transformer and sinkhorn tokenizer for sewer defect classification,” Au- tomation in Construction , vol. 144, p. 104 614, 2022, ISSN : 0926-5805. DOI: https : / / doi . org / 10 . 1016 / j . a...

    2022

  4. [12]

    Learning to merge tokens in vision transformers,

    C. Renggli, A. S. Pinto, N. Houlsby, B. Mustafa, J. Puigcerver, and C. Riquelme, “Learning to merge tokens in vision transformers,” arXiv preprint arXiv:2202.12015, 2022

    2022 arXiv

  5. [13]

    Learning to merge tokens via decoupled embedding for efficient vision transformers,

    D. H. Lee and S. Hong, “Learning to merge tokens via decoupled embedding for efficient vision transformers,” in Conference on Neural Information Processing Sys- tems, 2024

    2024

  6. [14]

    Token fusion: Bridging the gap between token pruning and token merging,

    M. Kim, S. Gao, Y .-C. Hsu, Y . Shen, and H. Jin, “Token fusion: Bridging the gap between token pruning and token merging,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , 2024, pp. 1383–1392

    2024

  7. [15]

    Diffrate: Differentiable compression rate for efficient vision transformers,

    M. Chen, W. Shao, P. Xu, et al., “Diffrate: Differentiable compression rate for efficient vision transformers,” in Proceedings of the IEEE/CVF international conference on computer vision , 2023, pp. 17 164–17 174

    2023

  8. [16]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, et al. , “An image is worth 16x16 words: Transformers for image recognition at scale,” ICLR, 2021

    2021

  9. [17]

    End-to-end object detection with transformers,

    N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kir- illov, and S. Zagoruyko, “End-to-end object detection with transformers,” inEuropean conference on computer vision, Springer, 2020, pp. 213–229

    2020

  10. [18]

    Vivit: A video vision transformer,

    A. Arnab, M. Dehghani, G. Heigold, C. Sun, M. Lu ˇci´c, and C. Schmid, “Vivit: A video vision transformer,” in Proceedings of the IEEE/CVF international conference on computer vision , 2021, pp. 6836–6846

    2021

  11. [19]

    Learn- ing transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, et al. , “Learn- ing transferable visual models from natural language supervision,” in International conference on machine learning, PmLR, 2021, pp. 8748–8763

    2021

  12. [20]

    Autoformer: Searching transformers for visual recognition,

    M. Chen, H. Peng, J. Fu, and H. Ling, “Autoformer: Searching transformers for visual recognition,” in Pro- ceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 12 270–12 280

    2021

  13. [21]

    Elasticvit: Conflict-aware supernet training for deploying fast vi- sion transformer on diverse mobile devices,

    C. Tang, L. L. Zhang, H. Jiang, et al. , “Elasticvit: Conflict-aware supernet training for deploying fast vi- sion transformer on diverse mobile devices,” in Pro- ceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 5829–5840

    2023

  14. [22]

    Revisit multimodal meta-learning through the lens of multi-task learning,

    M. Abdollahzadeh, T. Malekzadeh, and N. M. Cheung, “Revisit multimodal meta-learning through the lens of multi-task learning,” in Advances in Neural Information Processing Systems (NeurIPS) , vol. 34, 2021

    2021

  15. [23]

    Vct: A video compression transformer,

    F. Mentzer, G. Toderici, D. Minnen, et al. , “Vct: A video compression transformer,” in Advances in Neural Information Processing Systems (NeurIPS) , 2022

    2022

  16. [24]

    Highly parallel rate-distortion optimized intra-mode decision on multicore graphics processors,

    N. M. Cheung, O. C. Au, M. C. Kung, P. H. W. Wong, and C. H. Liu, “Highly parallel rate-distortion optimized intra-mode decision on multicore graphics processors,” IEEE Transactions on Circuits and Systems for Video Technology, 2009

    2009

  17. [25]

    On-device scalable image-based localization via prioritized cascade 5 search and fast one-many ransac,

    N.-T. Tran, D.-K. Le Tan, A.-D. Doan, et al., “On-device scalable image-based localization via prioritized cascade 5 search and fast one-many ransac,” IEEE Transactions on Image Processing, vol. 28, no. 4, pp. 1675–1690, 2018

    2018

  18. [26]

    On accelerating edge ai: Optimiz- ing resource-constrained environments,

    J. Sander, A. Cohen, V . R. Dasari, B. Venable, and B. Jalaian, “On accelerating edge ai: Optimiz- ing resource-constrained environments,” arXiv preprint arXiv:2501.15014, 2025

    2025 arXiv

  19. [27]

    A survey on efficient inference for large language models,

    Z. Zhou, X. Ning, K. Hong, et al., “A survey on efficient inference for large language models,” arXiv preprint arXiv:2404.14294, 2024

    2024 arXiv

  20. [28]

    Rlrc: Reinforcement learning- based recovery for compressed vision-language-action models,

    Y . Chen and X. Li, “Rlrc: Reinforcement learning- based recovery for compressed vision-language-action models,” arXiv preprint arXiv:2506.17639 , 2025

    2025 arXiv

  21. [29]

    Nvit: Vision transformer compression and parameter redistribution,

    H. Yang, H. Yin, P. Molchanov, H. Li, and J. Kautz, “Nvit: Vision transformer compression and parameter redistribution,” 2021

    2021

  22. [30]

    Width & depth pruning for vision transformers,

    F. Yu, K. Huang, M. Wang, Y . Cheng, W. Chu, and L. Cui, “Width & depth pruning for vision transformers,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, 2022, pp. 3143–3151

    2022

  23. [31]

    Revisiting label smoothing and knowledge distillation compatibility: What was missing?

    K. Chandrasegaran, N. T. Tran, Y . Zhao, and N. M. Cheung, “Revisiting label smoothing and knowledge distillation compatibility: What was missing?” In Pro- ceedings of the 39th International Conference on Ma- chine Learning (ICML) , 2022

    2022

  24. [32]

    Distilling the knowledge in a neural network,

    G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,” arXiv preprint arXiv:1503.02531, 2015

    2015 arXiv

  25. [33]

    Searching the search space of vision transformer,

    M. Chen, K. Wu, B. Ni, et al. , “Searching the search space of vision transformer,” Advances in Neural In- formation Processing Systems , vol. 34, pp. 8714–8726, 2021

    2021

  26. [34]

    Nasvit: Neural architecture search for efficient vision transformers with gradient conflict-aware supernet training,

    C. Gong and D. Wang, “Nasvit: Neural architecture search for efficient vision transformers with gradient conflict-aware supernet training,” ICLR Proceedings 2022, 2022

    2022

  27. [35]

    Vitas: Vision transformer architecture search,

    X. Su, S. You, J. Xie, et al., “Vitas: Vision transformer architecture search,” in European Conference on Com- puter Vision, Springer, 2022, pp. 139–157

    2022

  28. [37]

    Spvit: Enabling faster vision transformers via latency-aware soft token pruning,

    Z. Kong, P. Dong, X. Ma, et al. , “Spvit: Enabling faster vision transformers via latency-aware soft token pruning,” in European conference on computer vision , Springer, 2022, pp. 620–640

    2022

  29. [38]

    Token pooling in vision transformers for image classification,

    D. Marin, J.-H. R. Chang, A. Ranjan, A. Prabhu, M. Rastegari, and O. Tuzel, “Token pooling in vision transformers for image classification,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Jan. 2023, pp. 12–21

    2023

  30. [39]

    Not all tokens are equal: Human-centric visual analysis via token cluster- ing transformer,

    W. Zeng, S. Jin, W. Liu, et al. , “Not all tokens are equal: Human-centric visual analysis via token cluster- ing transformer,” in Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition , 2022, pp. 11 101–11 111. 6

    2022

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.