Pith. sign in

REVIEW 10 cited by

AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognition

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2205.13535 v3 pith:GG32WAEG submitted 2022-05-26 cs.CV

Shoufa Chen , Chongjian Ge , Zhan Tong , Jiangliu Wang , Yibing Song , Jue Wang , Ping Luo This is my paper · ORCID
classification cs.CV
keywords adaptformerdifferentrecognitiontasksvisualadaptimageparameters
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Pretraining Vision Transformers (ViTs) has achieved great success in visual recognition. A following scenario is to adapt a ViT to various image and video recognition tasks. The adaptation is challenging because of heavy computation and memory storage. Each model needs an independent and complete finetuning process to adapt to different tasks, which limits its transferability to different visual domains. To address this challenge, we propose an effective adaptation approach for Transformer, namely AdaptFormer, which can adapt the pre-trained ViTs into many different image and video tasks efficiently. It possesses several benefits more appealing than prior arts. Firstly, AdaptFormer introduces lightweight modules that only add less than 2% extra parameters to a ViT, while it is able to increase the ViT's transferability without updating its original pre-trained parameters, significantly outperforming the existing 100\% fully fine-tuned models on action recognition benchmarks. Secondly, it can be plug-and-play in different Transformers and scalable to many visual tasks. Thirdly, extensive experiments on five image and video datasets show that AdaptFormer largely improves ViTs in the target domains. For example, when updating just 1.5% extra parameters, it achieves about 10% and 19% relative improvement compared to the fully fine-tuned models on Something-Something~v2 and HMDB51, respectively. Code is available at https://github.com/ShoufaChen/AdaptFormer.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Streaming Detection of Queried Event Start

    cs.CV 2024-12 conditional novelty 7.0 of 10

    A new benchmark and task for detecting the start of a natural-language-described event in streaming egocentric video, with new metrics and adapter-based baselines.

  2. SIMPLER: Efficient Foundation Model Adaptation via Similarity-Guided Layer Pruning for Earth Observation

    cs.CV 2026-03 conditional novelty 6.0 of 10

    CKA-based pre-fine-tuning layer pruning selects redundant ViT depth on unlabeled EO task data, cutting up to ~79% parameters while retaining most task performance and speeding both train and inference.

  3. Exploring the Rashomon Set for Concept-Based Models

    cs.LG 2025-11 conditional novelty 6.0 of 10

    A shared frozen backbone plus per-model LoRA adapters and a concept-diversity loss trains a set of accurate CBMs that reason through different concepts.

  4. Densely Connected Parameter-Efficient Tuning for Referring Image Segmentation

    cs.CV 2025-01 conditional novelty 6.0 of 10

    DETRIS uses dense mixtures of convolutions and cross-attention adapters to tune a frozen DINOv2/CLIP pair, achieving top reported IoU on three referring image segmentation benchmarks while updating only a small fracti...

  5. CM3T: Framework for Efficient Multimodal Learning for Inhomogeneous Interaction Datasets

    cs.CV 2025-01 conditional novelty 6.0 of 10

    CM3T shows that multi-head vision adapters plus cross-attention adapters can adapt frozen supervised-pretrained video transformers with a fraction of the trainable parameters of full fine-tuning.

  6. EDTformer: An Efficient Decoder Transformer for Visual Place Recognition

    cs.CV 2024-12 conditional novelty 6.0 of 10

    A decoder transformer with learnable queries plus a low-rank parallel adapter for frozen DINOv2 achieves state-of-the-art visual place recognition on multiple benchmarks with reduced training memory.

  7. ER-LoRA: Effective-Rank Guided Adaptation for Weather-Generalized Depth Estimation

    cs.CV 2025-08 conditional novelty 5.0 of 10

    Tuning only 8.7M parameters of a frozen DINOv2 on daytime data is reported to beat prior PEFT, full fine-tuning, synthetic-data depth methods, and Depth Anything V2 on zero-shot adverse-weather benchmarks.

  8. Swin-TUNA : A Novel PEFT Approach for Accurate Food Image Segmentation

    cs.CV 2025-07 conditional novelty 5.0 of 10

    Swin-TUNA inserts layer-dependent depthwise-convolution adapters into a frozen Swin-L backbone and reports 50.56 mIoU on FoodSeg103 and 74.94 mIoU on UECFoodPix Complete with 8.13M trainable parameters.

  9. Visual Instance-aware Prompt Tuning

    cs.CV 2025-07 conditional novelty 5.0 of 10

    ViaPT generates instance-aware prompts per image, fuses them with dataset-level prompts, and applies PCA compression to outperform VPT-Deep and other PEFT baselines on FGVC, HTA, and VTAB-1k.

  10. Enhancing Parameter-Efficient Fine-Tuning of Vision Transformers through Frequency-Based Adaptation

    cs.CV 2024-11 reject novelty 5.0 of 10

    FreqFit is a frequency-domain filter module that, when inserted between ViT blocks, improves the accuracy of existing PEFT methods on most but not all evaluated benchmarks.

Pith tools