RECORD
UP-Person: Unified Parameter-Efficient Transfer Learning for Text-based Person Retrieval
Computer Vision and Pattern Recognition · Submitted 14 Apr 2025
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim Extensive experimental results demonstrate that our UP-Person achieves state-of-the-art results across various person retrieval datasets, including CUHK-PEDES, ICFG-PEDES and RSTPReid while merely fine-tuning 4.7\% parameters.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
Text-based Person Retrieval (TPR) as a multi-modal task, which aims to retrieve the target person from a pool of candidate images given a text description, has recently garnered considerable attention due to the progress of contrastive visual-language pre-trained model. Prior works leverage pre-trained CLIP to extract person visual and textual features and fully fine-tune the entire network, which have shown notable performance improvements compared to uni-modal pre-training models. However, full-tuning a large model is prone to overfitting and hinders the generalization ability. In this paper, we propose a novel Unified Parameter-Efficient Transfer Learning (PETL) method for Text-based Person Retrieval (UP-Person) to thoroughly transfer the multi-modal knowledge from CLIP. Specifically, UP-Person simultaneously integrates three lightweight PETL components including Prefix, LoRA and Adapter, where Prefix and LoRA are devised together to mine local information with task-specific information prompts, and Adapter is designed to adjust global feature representations. Additionally, two vanilla submodules are optimized to adapt to the unified architecture of TPR. For one thing, S-Prefix is proposed to boost attention of prefix and enhance the gradient propagation of prefix tokens, which improves the flexibility and performance of the vanilla prefix. For another thing, L-Adapter is designed in parallel with layer normalization to adjust the overall distribution, which can resolve conflicts caused by overlap and interaction among multiple submodules. Extensive experimental results demonstrate that our UP-Person achieves state-of-the-art results across various person retrieval datasets, including CUHK-PEDES, ICFG-PEDES and RSTPReid while merely fine-tuning 4.7\% parameters. Code is available at https://github.com/Liu-Yating/UP-Person.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | Text-based Person Retrieval (TPR) as a multi-modal task, which aims to retrieve the target person from a pool of candidate images given a text description, has recently garnered considerable attention due to the progress of contrastive visual-language pre-trained model. | background 1.00 | 0.05 | 0.02 | asserted 1.00 |
| s1 | Prior works leverage pre-trained CLIP to extract person visual and textual features and fully fine-tune the entire network, which have shown notable performance improvements compared to uni-modal pre-training models. | background 1.00 | 0.04 | 0.02 | asserted 1.00 |
| s2 | However, full-tuning a large model is prone to overfitting and hinders the generalization ability. | gap 0.54 | 0.08 | 0.03 | asserted 0.75 |
| s3 | In this paper, we propose a novel Unified Parameter-Efficient Transfer Learning (PETL) method for Text-based Person Retrieval (UP-Person) to thoroughly transfer the multi-modal knowledge from CLIP. | contribution 0.99 | 0.86 | 0.10 | asserted 0.98 |
| s4 | Specifically, UP-Person simultaneously integrates three lightweight PETL components including Prefix, LoRA and Adapter, where Prefix and LoRA are devised together to mine local information with task-specific information prompts, and Adapter is designed to adjust global feature representations. | method 0.94 | 0.92 | 0.07 | asserted 1.00 |
| s5 | Additionally, two vanilla submodules are optimized to adapt to the unified architecture of TPR. | method 0.92 | 0.84 | 0.03 | asserted 0.97 |
| s6 | For one thing, S-Prefix is proposed to boost attention of prefix and enhance the gradient propagation of prefix tokens, which improves the flexibility and performance of the vanilla prefix. | method 0.85 | 0.89 | 0.04 | asserted 0.99 |
| s7 | For another thing, L-Adapter is designed in parallel with layer normalization to adjust the overall distribution, which can resolve conflicts caused by overlap and interaction among multiple submodules. | method 0.92 | 0.89 | 0.03 | asserted 0.79 |
| s8 | Extensive experimental results demonstrate that our UP-Person achieves state-of-the-art results across various person retrieval datasets, including CUHK-PEDES, ICFG-PEDES and RSTPReid while merely fine-tuning 4.7\% parameters. | result 1.00 | 0.97 | 0.84 | asserted 1.00 |
| s9 | Code is available at https://github.com/Liu-Yating/UP-Person. | contribution 0.49 | 0.52 | 0.02 | asserted 0.92 |
What was examined, and what was not
- arXiv listing abstract v1
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
Text-based Person Retrieval (TPR) as a multi-modal task, which aims to retrieve the target person from a pool of candidate images given a text description, has recently garnered considerable attention due to the progress of contrastive visual-language pre-trained model. Prior works leverage pre-trained CLIP to extract person visual and textual features and fully fine-tune the entire network, which have shown notable performance improvements compared to uni-modal pre-training models. However, full-tuning a large model is prone to overfitting and hinders the generalization ability. In this paper, we propose a novel Unified Parameter-Efficient Transfer Learning (PETL) method for Text-based Person Retrieval (UP-Person) to thoroughly transfer the multi-modal knowledge from CLIP. Specifically, UP-Person simultaneously integrates three lightweight PETL components including Prefix, LoRA and Adapter, where Prefix and LoRA are devised together to mine local information with task-specific information prompts, and Adapter is designed to adjust global feature representations. Additionally, two vanilla submodules are optimized to adapt to the unified architecture of TPR. For one thing, S-Prefix is proposed to boost attention of prefix and enhance the gradient propagation of prefix tokens, which improves the flexibility and performance of the vanilla prefix. For another thing, L-Adapter is designed in parallel with layer normalization to adjust the overall distribution, which can resolve conflicts caused by overlap and interaction among multiple submodules. Extensive experimental results demonstrate that our UP-Person achieves state-of-the-art results across various person retrieval datasets, including CUHK-PEDES, ICFG-PEDES and RSTPReid while merely fine-tuning 4.7\% parameters. Code is available at https://github.com/Liu-Yating/UP-Person.
Human review
How the literature uses it
What this paper does with the arXiv papers it cites
What it cites · 20 sentences without a confident label
Uses the method or tool of 1 paper
-
For the input text $T$, we tokenize the input description to embeddings $f$ by a simple tokenizer with a 49152 vocab size [ref] .
Neural Machine Translation of Rare Words with Subword Units · Feature Extraction · 0.83
Compares with 2 papers
-
As shown in Table [ref], UP-Person on MSCOCO and Flicker exceeds other methods of similar scale with millions of parameters by a large margin, such as IMEB [ref] and DSMD [ref] .
Dynamic Self-adaptive Multiscale Distillation from Pre-trained Multimodal Large Model for Efficient Cross-modal Representation Learning · Coarse-grained Text-to-Image Retrieval · 1.00
-
Table [ref] shows the effectiveness of LN-tuning [ref] , vanilla adapter and our L-Adapter in UP-Person, and further explores the sequential and parallel connection.
Parameter-Efficient Tuning on Layer Normalization for Pre-trained Language Models · Analysis of L-Adapter. · 0.89
Cites as background 10 papers
-
LoRA [ref] incorporates low-rank trainable matrices into transformer layers with the aim of providing an approximation to weight updates.
LoRA: Low-Rank Adaptation of Large Language Models · Preliminaries · 0.97
-
PETL has emerged as a viable strategy to compensate for the above disadvantages of full-tuning, which can be broadly categorized into three types [ref] : additive fine-tuning, reparameterized fine-tuning and selective fine-tuning.
Parameter-Efficient Fine-Tuning for Large Models: A Comprehensive Survey · Parameter-Efficient Transfer Learning. · 0.87
-
Cross-modal prompt called MaPLe [ref] and cross-modal adapter [ref] are proposed in both vision and language branches and further achieve cross-modal interactions.
Cross-Modal Adapter for Vision-Language Retrieval · Parameter-Efficient Transfer Learning. · 0.86
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.