Pith. sign in

REVIEW 8 cited by

AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognition

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2205.13535 v3 pith:GG32WAEG submitted 2022-05-26 cs.CV

Shoufa Chen , Chongjian Ge , Zhan Tong , Jiangliu Wang , Yibing Song , Jue Wang , Ping Luo This is my paper
classification cs.CV
keywords adaptformerdifferentrecognitiontasksvisualadaptimageparameters
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Pretraining Vision Transformers (ViTs) has achieved great success in visual recognition. A following scenario is to adapt a ViT to various image and video recognition tasks. The adaptation is challenging because of heavy computation and memory storage. Each model needs an independent and complete finetuning process to adapt to different tasks, which limits its transferability to different visual domains. To address this challenge, we propose an effective adaptation approach for Transformer, namely AdaptFormer, which can adapt the pre-trained ViTs into many different image and video tasks efficiently. It possesses several benefits more appealing than prior arts. Firstly, AdaptFormer introduces lightweight modules that only add less than 2% extra parameters to a ViT, while it is able to increase the ViT's transferability without updating its original pre-trained parameters, significantly outperforming the existing 100\% fully fine-tuned models on action recognition benchmarks. Secondly, it can be plug-and-play in different Transformers and scalable to many visual tasks. Thirdly, extensive experiments on five image and video datasets show that AdaptFormer largely improves ViTs in the target domains. For example, when updating just 1.5% extra parameters, it achieves about 10% and 19% relative improvement compared to the fully fine-tuned models on Something-Something~v2 and HMDB51, respectively. Code is available at https://github.com/ShoufaChen/AdaptFormer.

Discussion (0). Sign in to comment.

Forward citations

Cited by 8 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. SLAD : Shared LoRA Adapters for Task Specific Distillation

    cs.CV 2026-05 unverdicted novelty 6.0 of 10

    SLAD uses shared LoRA adapters in joint training to align teacher-student features, boosting both models' performance and halving training time versus fine-tuning in distillation.

  2. Hystar: Hypernetwork-driven Style-adaptive Retrieval via Dynamic SVD Modulation

    cs.CV 2026-05 unverdicted novelty 6.0 of 10

    Hystar adapts CLIP-like models to unseen query styles by generating per-input singular-value perturbations with a hypernetwork for attention layers and a new StyleNCE contrastive loss.

  3. Is Prompt Selection Necessary for Task-Free Online Continual Learning?

    cs.LG 2026-04 unverdicted novelty 6.0 of 10

    SinglePrompt achieves state-of-the-art results in task-free online continual learning by replacing prompt selection with a single prompt per attention block, cosine-based classifier logits, and masking unexposed classes.

  4. SIMPLER: Efficient Foundation Model Adaptation via Similarity-Guided Layer Pruning for Earth Observation

    cs.CV 2026-03 conditional novelty 6.0 of 10

    CKA-based pre-fine-tuning layer pruning selects redundant ViT depth on unlabeled EO task data, cutting up to ~79% parameters while retaining most task performance and speeding both train and inference.

  5. Exploring the Rashomon Set for Concept-Based Models

    cs.LG 2025-11 conditional novelty 6.0 of 10

    A shared frozen backbone plus per-model LoRA adapters and a concept-diversity loss trains a set of accurate CBMs that reason through different concepts.

  6. Robust Adaptation of Foundation Models with Black-Box Visual Prompting

    cs.CV 2024-07 unverdicted novelty 6.0 of 10

    BlackVIP adapts foundation models via a Coordinator for input-dependent visual prompts and SPSA-GC for gradient estimation, enabling robust transfer on 19 datasets with low memory use and a link to randomized smoothin...

  7. ER-LoRA: Effective-Rank Guided Adaptation for Weather-Generalized Depth Estimation

    cs.CV 2025-08 conditional novelty 5.0 of 10

    Tuning only 8.7M parameters of a frozen DINOv2 on daytime data is reported to beat prior PEFT, full fine-tuning, synthetic-data depth methods, and Depth Anything V2 on zero-shot adverse-weather benchmarks.

  8. Bridging the Generalization Gap in Adverse Weather Segmentation: A Training Recipe Perspective

    cs.CV 2026-05 unverdicted novelty 2.0 of 10

    A training recipe combining domain-adaptive fine-tuning, multi-source mixing, balanced sampling, and synthetic augmentations on SegMAN-S achieves 59.9% mIoU on the adverse weather test set with a 6.5-point validation-...

Pith tools