Pith. sign in

REVIEW 2 cited by

Alpha-CLIP: A CLIP Model Focusing on Wherever You Want

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2312.03818 v2 pith:BPHR6WND submitted 2023-12-06 cs.CV cs.AIcs.CLcs.LG

Zeyi Sun , Ye Fang , Tong Wu , Pan Zhang , Yuhang Zang , Shu Kong , Yuanjun Xiong , Dahua Lin
show 1 more author
Jiaqi Wang
This is my paper · ORCID
classification cs.CVcs.AIcs.CLcs.LG
keywords cliptasksalpha-clipimageimagesincludingmodelsrecognition
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Contrastive Language-Image Pre-training (CLIP) plays an essential role in extracting valuable content information from images across diverse tasks. It aligns textual and visual modalities to comprehend the entire image, including all the details, even those irrelevant to specific tasks. However, for a finer understanding and controlled editing of images, it becomes crucial to focus on specific regions of interest, which can be indicated as points, masks, or boxes by humans or perception models. To fulfill the requirements, we introduce Alpha-CLIP, an enhanced version of CLIP with an auxiliary alpha channel to suggest attentive regions and fine-tuned with constructed millions of RGBA region-text pairs. Alpha-CLIP not only preserves the visual recognition ability of CLIP but also enables precise control over the emphasis of image contents. It demonstrates effectiveness in various tasks, including but not limited to open-world recognition, multimodal large language models, and conditional 2D / 3D generation. It has a strong potential to serve as a versatile tool for image-related tasks.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. MatCLIP: Light- and Shape-Insensitive Assignment of PBR Material Models

    cs.CV 2025-01 conditional novelty 6.0 of 10

    MatCLIP learns a shape- and lighting-robust CLIP-based descriptor of PBR materials from 42 renderings per material and uses it to match materials to image regions, reaching 76.69% top-1 accuracy.

  2. ProKeR: A Kernel Perspective on Few-Shot Adaptation of Large Vision-Language Models

    cs.CV 2025-01 conditional novelty 6.0 of 10

    ProKeR treats CLIP cache models as Nadaraya-Watson estimators and fits a proximally regularized kernel ridge regression in an RKHS, reporting state-of-the-art training-free few-shot accuracy on 11 datasets.

Pith tools