A prototype-based few-shot classifier for multi-label images that combines word-embedding priors, loss-based local feature selection, and multi-modal attention, outperforming prior methods on COCO, PASCAL VOC, NUS-WIDE, and iMaterialist.
Efficient Vision-Language Pre-training by Cluster Masking
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
We propose a simple strategy for masking image patches during visual-language contrastive learning that improves the quality of the learned representations and the training speed. During each iteration of training, we randomly mask clusters of visually similar image patches, as measured by their raw pixel intensities. This provides an extra learning signal, beyond the contrastive training itself, since it forces a model to predict words for masked visual structures solely from context. It also speeds up training by reducing the amount of data used in each image. We evaluate the effectiveness of our model by pre-training on a number of benchmarks, finding that it outperforms other masking strategies, such as FLIP, on the quality of the learned representation.
citation-role summary
background 1
citation-polarity summary
fields
cs.CV 1years
2024 1verdicts
CONDITIONAL 1roles
background 1polarities
unclear 1representative citing papers
citing papers explorer
-
Modeling Multi-modal Cross-interaction for Multi-label Few-shot Image Classification Based on Local Feature Selection
A prototype-based few-shot classifier for multi-label images that combines word-embedding priors, loss-based local feature selection, and multi-modal attention, outperforming prior methods on COCO, PASCAL VOC, NUS-WIDE, and iMaterialist.