VFE-TPS adds text-guided masked image modeling and identity-supervised feature calibration to CLIP and reports state-of-the-art Rank-1 accuracy on three text-based person search benchmarks, though not above a cited RaSa result.
CLIP-based Synergistic Knowledge Transfer for Text-based Person Retrieval
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
Text-based Person Retrieval (TPR) aims to retrieve the target person images given a textual query. The primary challenge lies in bridging the substantial gap between vision and language modalities, especially when dealing with limited large-scale datasets. In this paper, we introduce a CLIP-based Synergistic Knowledge Transfer (CSKT) approach for TPR. Specifically, to explore the CLIP's knowledge on input side, we first propose a Bidirectional Prompts Transferring (BPT) module constructed by text-to-image and image-to-text bidirectional prompts and coupling projections. Secondly, Dual Adapters Transferring (DAT) is designed to transfer knowledge on output side of Multi-Head Attention (MHA) in vision and language. This synergistic two-way collaborative mechanism promotes the early-stage feature fusion and efficiently exploits the existing knowledge of CLIP. CSKT outperforms the state-of-the-art approaches across three benchmark datasets when the training parameters merely account for 7.4% of the entire model, demonstrating its remarkable efficiency, effectiveness and generalization.
citation-role summary
background 1
citation-polarity summary
fields
cs.CV 1years
2024 1verdicts
CONDITIONAL 1roles
background 1polarities
unclear 1representative citing papers
citing papers explorer
-
Enhancing Visual Representation for Text-based Person Searching
VFE-TPS adds text-guided masked image modeling and identity-supervised feature calibration to CLIP and reports state-of-the-art Rank-1 accuracy on three text-based person search benchmarks, though not above a cited RaSa result.