Pith. sign in

REVIEW 5 major objections 6 minor 57 references

Text to Image for Multi-Label Image Recognition with Joint Prompt-Adapter Learning

T0 review · 5 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read T2I-PAL claims that generating synthetic images from captions with a text-to-image model and learning prompts jointly with a shared adapter closes the modality gap in CLIP fine-tuning, enabling multi-label recognition without fully…

desk verdict A useful, incremental extension of TaI that swaps text embeddings for Stable Diffusion images and backs it with thorough ablations, but the headline 3.47% gain is not reconstructible from the paper's own tables. read the letter →

arxiv 2506.10575 v1 pith:ONR6ZCJF submitted 2025-06-12 cs.CV

Chun-Mei Feng , Kai Yu , Xinxing Xu , Salman Khan , Rick Siow Mong Goh , Wangmeng Zuo , Yong Liu This is my paper
classification cs.CV
keywords multi-labelimagerecognitiontext-to-imagesynthesismodalitygapprompttuningadapterlearningparameter-efficientfine-tuningCLIPStableDiffusion
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

T2I-PAL claims that the modality gap limiting text-as-image fine-tuning of CLIP can be bridged by generating synthetic images from captions with a pretrained text-to-image model rather than by using raw text features. The paper shows that with only captions and no fully annotated training images, the method outperforms state-of-the-art multi-label recognition baselines by 3.47% average mAP on MS-COCO, VOC2007, and NUS-WIDE. It also reports that one synthetic image per caption matches one original image in recognition performance, while six synthetic images per caption outperform the original images. The method freezes CLIP and trains only lightweight prompts and a shared adapter, so it can be dropped into existing CLIP pipelines and substantially cuts annotation cost. If correct, this makes caption-only supervision a practical route for multi-label image recognition.

What carries the argument

The load-bearing mechanism is the joint prompt-adapter module over frozen CLIP encoders. Global and local learnable prompts are appended to class tokens; the class-wise heatmap $h_{ij} = \exp(S_{ij}/\tau)/\sum_j \exp(S_{ij}/\tau)$ weights local similarities to give a robust class similarity $s'_i$, and the same weighting builds class-attended features $H_i$ that query a learnable prototype matrix $A \in \mathbb{R}^{C\times D}$ shared by the text and synthetic-image branches. The final local logit is $\tilde{s}'_i = \alpha q_i + s'_i$, blending CLIP's prior knowledge with adapter knowledge. The shared matrix is what lets the synthetic-image branch absorb information from the text branch, and the heatmap is what makes local features informative for multi-label prediction.

What would settle it

Train the same method on a caption set dominated by classes that text-to-image models tend to omit, such as small or rare objects, and compare per-class accuracy against the text-only baseline; if the omitted classes show no gain or a drop, the claim that synthetic images close the modality gap for multi-label recognition would be refuted.

Watch

Extended reading notes

Core claim

The paper's central claim is that for parameter-efficient fine-tuning of CLIP on multi-label recognition using only text captions, the bottleneck is not caption informativeness but the modality gap between text features and the image features seen at test time. Its proposed solution, T2I-PAL, feeds captions through Stable Diffusion to produce synthetic images, encodes those images with the frozen CLIP image encoder, and learns global and local prompts together with an adapter whose prototype matrix is shared between the text-caption branch and the synthetic-image branch. A class-wise heatmap aggregates local similarities and produces class-attended features for the adapter. The paper reports that this reduces the cosine gap between training features and prompted class embeddings from 0.3178 (text-as-image) to 0.6110, and translates into average mAP gains of 0.6, 1.5, and 0.8 points over the top baseline in zero-shot settings on MS-COCO, VOC2007, and NUS-WIDE, with a 3.47% average improvement across benchmarks and settings.

Load-bearing premise

The argument depends on synthetic images generated from captions carrying the same label-relevant visual information as real annotated images, so that classifiers trained on them transfer to real test images; this parity is demonstrated on only three datasets and with a single Stable Diffusion version.

Editorial extensions

If this is right

  • Multi-label classifiers can be trained from captions alone, removing the need for fully semantically annotated training images and lowering the annotation burden for large label sets.
  • Because CLIP's encoders stay frozen and the only new parameters are prompts and a small adapter, the method can be plugged into any existing CLIP-based pipeline without changing the pretrained model.
  • Increasing synthetic data volume alone improves accuracy, with six generated images per caption outperforming the original images, so the method's ceiling is partly set by generator diversity rather than annotation effort.
  • The same recipe transfers beyond multi-label recognition, with reported gains on Flowers102, Food101, and AWA2 suggesting it also helps single-label and attribute classification.
  • Under few-shot and partial-label settings, integrating T2I-PAL with existing methods such as CoOp and DualCoOp raises accuracy, including a zero-shot result that surpasses a 5-shot baseline.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper itself notes that it relies on a pretrained text-to-image model and does not address biases or fidelity limits in generated images; the open risk is thus generator weakness rather than caption or prompt design.
  • If the synthetic-to-real transfer holds beyond the three datasets tested, the practical bottleneck for supervised recognition shifts from image annotation to caption quality and text-to-image fidelity, and the claimed parity would likely degrade in specialized domains where generators are weak.
  • The paper's own failure cases show that text-to-image generation sometimes omits objects, so a strong testable extension is per-class analysis: classes the generator depicts reliably should drive the gains, while systematically omitted classes should show little or no improvement over text-only training.
  • The shared prototype matrix between the text and synthetic-image branches suggests a general recipe for cross-modal parameter-efficient tuning, exchanging a lightweight learned representation between modalities rather than aligning full feature spaces.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes T2I-PAL, a parameter-efficient fine-tuning method for multi-label image recognition that avoids using fully annotated real training images. Text captions are used to generate synthetic images with Stable Diffusion; CLIP's image and text encoders are kept frozen, and only learnable prompts and a shared adapter are trained on the synthetic images and the original captions. The method is evaluated under zero-shot, few-shot, and partial-label settings on MS-COCO, VOC 2007, and NUS-WIDE, with additional experiments on single-label and attribute classification. The central claims are that replacing text features with features of synthetic images reduces the modality gap of text-as-image methods such as TaI-DPT, and that joint prompt-adapter learning yields an average improvement of 3.47% over top-ranked state-of-the-art methods.

Significance. If the empirical claims are reproducible, the paper makes a useful contribution: it shows that text-to-image synthesis can substitute for annotated training images in CLIP-based multi-label recognition, and the battery of experiments across zero-shot, few-shot, partial-label, and transfer settings is broad. The paper also contains several positive features that should be credited: detailed ablations on adapter placement, synthetic-data scale, caption source (including LLaMA-generated captions), and Stable Diffusion versions; explicit failure cases in Fig. 10; and a candid limitation statement in Section V. However, the headline improvement number is not auditable from the paper's own tables, and the modality-gap analysis is partly circular because it uses the same model that is being optimized. These issues do not invalidate the core idea but they do require substantial revision before the paper can be accepted.

major comments (5)
  1. [Abstract; Table I; Section IV-B] The headline claim of a 3.47% average improvement is not reproducible from the reported tables because no averaging protocol is defined. In Table I, the caption says the gains are over TaI-DPT, but the parenthetical gains 0.6/1.5/0.8 match the gaps over PVP on MS-COCO and VOC 2007 and over Dong et al. on NUS-WIDE; the gains over TaI-DPT would be 6.3/3.2/0.9. Table IV's +T2I-PAL versus +TaI-DPT gains average 5.5 on MS-COCO and 2.94 on VOC 2007, Table XII's RN101 gains average 3.77, and Table II's partial-label gains average roughly 0.5. No stated rule over these heterogeneous settings yields 3.47. The authors should either specify the exact baseline and averaging protocol or remove/qualify the headline number.
  2. [Section IV-G; Table XII] Table XII's column order is inconsistent with the text and with Tables I and IV. The text states that on ResNet-50 T2I-PAL obtains 88.8 on VOC 2007, 66.1 on MS-COCO, and 45.5 on NUS-WIDE, and on ResNet-101 obtains 91.5/71.4/47.4, but the table prints these values under the headers MS-COCO, VOC 2007, NUS-WIDE, with the first two columns effectively swapped. As printed, the table shows MS-COCO values of 88.8 and 91.5, contradicting the zero-shot results in Table I. This table must be corrected before the visual-encoder comparison can be evaluated.
  3. [Section IV-D; Table VI] The modality-gap analysis is not an independent measurement. The cosine similarities in Table VI are computed with the same CLIP encoders, the same learned prompts, and the same shared adapter that are optimized to maximize similarity between synthetic-image features and text-prompt features. The increase from 0.3178 to 0.6110 is therefore expected from the training objective and does not by itself demonstrate a reduced gap between synthetic and real image distributions. The authors should either use a fixed, untrained reference encoder or report the similarity of real test images after adapter training; the mechanistic interpretation is currently not supported, even though the real-image test accuracy remains a valid outcome measure.
  4. [All experimental tables] No standard deviations, confidence intervals, or significance tests are reported for any experiment. Several claimed improvements are very small, for example NUS-WIDE at 20% in Table II (56.7 vs. 56.4) and the zero-shot NUS-WIDE comparison in Table I (47.4 vs. 46.6). Without multiple seeds or an explicit statement about the number of runs, the language "significantly outperforms" and "consistently improves" is not statistically supported. Please provide means over multiple runs with uncertainty intervals, or state that the differences are not tested for significance.
  5. [Section IV-E; Section IV-F; Tables VIII-X] There are internal numerical inconsistencies in the ablations. Section IV-E states that with one synthetic image per caption the method reaches 91.1 on VOC 2007, but Table VIII (Ours(S.I)) and Table IX (Ours(1xS.I)) both report 90.8. In Section IV-F the text says that doubling the text captions "consistently improves" classification performance, but Table X reports 91.8 for T2I-PAL and 91.5 for T2I-PAL w/2x on VOC 2007, which is a decrease. These numbers need to be reconciled or corrected.
minor comments (6)
  1. [Abstract; Fig. 2 caption] There are several language and typographical issues, including "allow to direct leverage texts as images" in the abstract and "pertained CLIP" in the Fig. 2 caption; both should be corrected to "pre-trained CLIP" or similar.
  2. [Section IV-A; Table I caption] The phrase "top-1 ranked baseline" is ambiguous because the best baseline differs per dataset and per setting. Please define explicitly which method is the reference for each table and for the abstract's average.
  3. [References [52]; Section IV-A] Reference [52] is listed as "Tai++: Text as image for multi-label image classification by co-learning transferable prompt," but the manuscript refers to this method as "PVP" throughout. Please clarify the name and citation.
  4. [Section IV-G] The subsection heading "Effect of PEFT" is misleading: the subsection analyzes adapter placement and visual encoder choice, not parameter-efficient fine-tuning as a general concept. A more specific heading such as "Effect of Adapter and Visual Encoder" would be clearer.
  5. [Section III-B; Eq. (5)] The notation for the learnable matrix is inconsistent: the text describes an initialized matrix of size D x C, while Eq. (5) treats A as C x D and uses H A^T. Please standardize the dimensions and notation.
  6. [Section IV-B] In the few-shot discussion, "zero-shot TaI-DP" should read "TaI-DPT" for consistency with the rest of the paper.

Circularity Check

1 steps flagged · score 2.0 of 10

No load-bearing circularity: test accuracy is measured on real images; only the modality-gap ablation reuses the training objective as its metric.

  1. self definitional [Section IV-D, Table VI; compared with Section III-C Eq. (7)]
    "More importantly, after applying the adapter, the domain gap is further reduced, with the similarity between synthetic training images and Text.pro + CLS improving from 0.3178 to 0.6110 compared to TaI. These results clearly demonstrate that synthetic images and the shared adapter effectively mitigate the modality gap."

    The cosine-similarity metric in Table VI is not an independent probe: the adapter and prompts are trained with the ranking loss in Eq. (7) on the same global/local similarities s_i=<f_g,G_i> and es'_i from Eqs. (2) and (6). Reporting that the similarity between synthetic training images and Text.pro+CLS rises from 0.3178 to 0.6110 after training is therefore reporting the training objective itself, so the modality-gap reduction shown in this table is partly by construction. The central test-accuracy claim, however, is measured on real images against external baselines and does not reduce to this metric.

full rationale

The paper's derivation chain is largely self-contained: the central quantitative claims are mAP values on real test images for MS-COCO, VOC2007, and NUS-WIDE, compared against published baselines, so the headline improvement is an external empirical measurement rather than a fitted prediction. The only self-referential element is the modality-gap analysis in Section IV-D, where the chosen cosine-similarity metric is the same objective used to train the adapter and prompts; this makes that particular ablation a training-loss echo, but it does not by itself force the test-accuracy results. The abstract's '3.47% average improvement' is hard to reconstruct from the tables because the comparison baseline shifts across tables (e.g., Table I versus Table IV), but that is an auditability or reporting issue rather than a circularity. Self-citations to prior work by the same authors appear in related work and in the TaI-DPT baseline; they are not used to invoke a uniqueness theorem or to forbid alternatives, and the method is evaluated against independent external methods. Overall, no load-bearing circularity is present.

Assumptions & free parameters 7 free parameters · 4 assumptions · 0 invented entities

No new physical entities are introduced; all learnable components (prompts, adapter, prototypes) are parameters of the model. The free parameters are mostly standard hyperparameters chosen empirically.

free parameters (7)
  • gamma = 0.2
    Trade-off weight between synthetic image loss and text loss; set empirically (Section IV-A).
  • alpha = 1
    Residual ratio in adapter logits (Eq. 6); chosen by hyperparameter sweep (Section IV-G).
  • beta = 3.5
    Modulating hyperparameter for affinity sharpness (Eq. 5); set empirically (Section IV-G).
  • eta = 1
    Margin in ranking loss (Eq. 7); set empirically.
  • tau = 0.02
    Temperature in softmax aggregation (Eq. 3); set empirically.
  • M (prompt length) = 16
    Number of learnable prompt tokens; chosen by hand.
  • synthetic_data_scale = 6x
    Number of synthetic images per caption and total dataset size; selected based on Fig. 3b.
assumptions (4)
  • domain assumption CLIP's pretrained text and image encoders provide aligned features that can be fine-tuned with prompts and adapters
    The entire method relies on CLIP's pretrained representations; this is standard practice but not formally justified.
  • domain assumption Stable Diffusion generates images that faithfully represent the objects in the captions
    The method's effectiveness depends on this; the paper acknowledges failure cases in Fig. 10.
  • domain assumption The noun filter correctly maps caption words to target labels
    Follows TaI-DPT; errors could introduce label noise.
  • standard math Adam optimizer and standard training setup converge reliably
    No proof of convergence; typical for deep learning papers.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Text to Image for Multi-Label Image Recognition with Joint Prompt-Adapter Learning." pith.science (2026). https://pith.science/paper/ONR6ZCJF

@misc{pith2026250610575,
  author       = {Pith},
  title        = {Pith review of: Text to Image for Multi-Label Image Recognition with Joint Prompt-Adapter Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ONR6ZCJF}},
  note         = {Machine review of arXiv:2506.10575}
}
read the original abstract

Benefited from image-text contrastive learning, pre-trained vision-language models, e.g., CLIP, allow to direct leverage texts as images (TaI) for parameter-efficient fine-tuning (PEFT). While CLIP is capable of making image features to be similar to the corresponding text features, the modality gap remains a nontrivial issue and limits image recognition performance of TaI. Using multi-label image recognition (MLR) as an example, we present a novel method, called T2I-PAL to tackle the modality gap issue when using only text captions for PEFT. The core design of T2I-PAL is to leverage pre-trained text-to-image generation models to generate photo-realistic and diverse images from text captions, thereby reducing the modality gap. To further enhance MLR, T2I-PAL incorporates a class-wise heatmap and learnable prototypes. This aggregates local similarities, making the representation of local visual features more robust and informative for multi-label recognition. For better PEFT, we further combine both prompt tuning and adapter learning to enhance classification performance. T2I-PAL offers significant advantages: it eliminates the need for fully semantically annotated training images, thereby reducing the manual annotation workload, and it preserves the intrinsic mode of the CLIP model, allowing for seamless integration with any existing CLIP framework. Extensive experiments on multiple benchmarks, including MS-COCO, VOC2007, and NUS-WIDE, show that our T2I-PAL can boost recognition performance by 3.47% in average above the top-ranked state-of-the-art methods.

Figures

Figures reproduced from arXiv: 2506.10575 by the authors.

Figure 1
Figure 1. Comparison of existing prompt tuning methods and our T2I-PAL for MLR. (a) DualCoOp [44], which requires a substantial set of annotated images to learn the prompts, thereby being costly in annotation. (b) TaI-DPT [18], which solely leverages a set of text captions to learn the prompts, but suffers from the modality gap issue. In comparison, we present (c) T2I-PAL, where the pre-trained text-to-image generation model … view at source ↗
Figure 2
Figure 2. Overall pipeline of our proposed T2I-PAL. (a) Using pre-trained text-to-image generation models [39] to generate synthesized images from text captions and jointly learning prompt-adapter without modifying the inherent mode of the pertained CLIP. (b) Sharing adapter in the text caption branch to further enhance the classification performance. many applications, e.g., high-resolution image synthesis [27], [39], text-t… view at source ↗
Figure 3
Figure 3. (a) Analysis with regard to the different values of γ, where the larger the value of the γ, the larger the proportion of the synthetic image in our method, vice versa; (d) Ablations on the different size of synthetic data on our method. TABLE V COMPARISON WITH STATE-OF-THE-ART METHODS FOR SINGLE-LABEL CLASSIFICATION (FLOWERS102, FOOD101) AND MULTI-LABEL ATTRIBUTE CLASSIFICATION (AWA2). Method ZSCLIP TaI-DPT T2I-PAL … view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Analysis of the quality of text captions and synthetic images influence our method on the three datasets. TABLE IX ABLATION STUDIES OF ELIMINATION OF THE MODALITY GAP. Methods MS-COCO VOC 2007 NUS-WIDE Ours(1×S.I) 70.5 90.8 46.8 Ours(O.I-40%) 70.5 90.9 46.8 Ours(O.I-80…
Figure 5
Figure 5. Figure 5: (a) Illustration of leveraging Llama to generate text captions; (b) Examples of VOC captions with the selected labels. As a comparison, Llama can also [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: Zero-shot performance of our method at different CFG scales. [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]
Figure 7
Figure 7. Figure 7: Exemplars of the complementarity of TaI-DPT [18] and T2I, where and refer to the correct and wrong predictions of each class, respectively. TABLE XIV ABLATION STUDIES OF EQ. (3) AND EQ. (6). Variation Mean Max MS-COCO VOC-2007 NUS-WIDE Average T2I-PAL(Mean) ! − 71.0(0.…
Figure 8
Figure 8. Figure 8: Visualization of the synthetic image with their corresponding text description and original image. thereby reducing model performance. The ‘max’ operation highlights prominent areas but may neglect contributions from other regions, also resulting in decreased performan…
Figure 9
Figure 9. Figure 9: Analysis with regard to the different values of α, and β on the two datasets, i.e., VOC 2007 and MS-COCO, where (a) the smaller the value of α, the more prior knowledge needs to be acquired from the pre-trained CLIP’s visual encoder, vice versa; and (b) β controls the …
Figure 10
Figure 10. Figure 10: Failure cases of the synthetic image, where some objects are missing and marked in red. of the CLIP model and can be implanted into any CLIP model. Additionally, T2I-PAL combines both prompt tuning and adapter learning with the two modalities, thereby enhancing classi…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

57 extracted references · 35 canonical work pages

  1. [1]

    Flamingo: a visual language model for few- shot learning.Advances in Neural Information Processing Systems, 35:23716–23736, 2022

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few- shot learning.Advances in Neural Information Processing Systems, 35:23716–23736, 2022

    work page 2022

  2. [2]

    Laso: Label-set operations networks for multi-label few-shot learning

    Amit Alfassy, Leonid Karlinsky, Amit Aides, Joseph Shtok, Sivan Harary, Rogerio Feris, Raja Giryes, and Alex M Bronstein. Laso: Label-set operations networks for multi-label few-shot learning. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6548–6557, 2019

    work page 2019

  3. [3]

    Structured semantic transfer for multi-label recognition with partial labels

    Tianshui Chen, Tao Pu, Hefeng Wu, Yuan Xie, and Liang Lin. Structured semantic transfer for multi-label recognition with partial labels. In Proceedings of the AAAI conference on artificial intelligence, volume 36, pages 339–346, 2022

    work page 2022

  4. [4]

    Recurrent attentional reinforcement learning for multi-label image recognition

    Tianshui Chen, Zhouxia Wang, Guanbin Li, and Liang Lin. Recurrent attentional reinforcement learning for multi-label image recognition. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018

    work page 2018

  5. [6]

    Learning semantic-specific graph representation for multi-label image recognition

    Tianshui Chen, Muxin Xu, Xiaolu Hui, Hefeng Wu, and Liang Lin. Learning semantic-specific graph representation for multi-label image recognition. InProceedings of the IEEE/CVF international conference on computer vision, pages 522–531, 2019

    work page 2019

  6. [7]

    Multi- label image recognition with graph convolutional networks

    Zhao-Min Chen, Xiu-Shen Wei, Peng Wang, and Yanwen Guo. Multi- label image recognition with graph convolutional networks. InPro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5177–5186, 2019

    work page 2019

  7. [8]

    Stargan v2: Diverse image synthesis for multiple domains

    Yunjey Choi, Youngjung Uh, Jaejun Yoo, and Jung-Woo Ha. Stargan v2: Diverse image synthesis for multiple domains. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8188–8197, 2020

    2020

  8. [9]

    Nus-wide: a real-world web image database from national university of singapore

    Tat-Seng Chua, Jinhui Tang, Richang Hong, Haojie Li, Zhiping Luo, and Yantao Zheng. Nus-wide: a real-world web image database from national university of singapore. InProceedings of the ACM international conference on image and video retrieval, pages 1–9, 2009

    work page 2009

Show all 57 references
  1. [10]

    Variational prompt tuning improves generalization of vision-language models.arXiv preprint arXiv:2210.02390, 2022

    Mohammad Mahdi Derakhshani, Enrique Sanchez, Adrian Bulat, Victor Guilherme Turrisi da Costa, Cees GM Snoek, Georgios Tzimiropoulos, and Brais Martinez. Variational prompt tuning improves generalization of vision-language models.arXiv preprint arXiv:2210.02390, 2022

    arXiv

  2. [11]

    Learning a deep convnet for multi-label classification with partial labels

    Thibaut Durand, Nazanin Mehrasa, and Greg Mori. Learning a deep convnet for multi-label classification with partial labels. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 647–657, 2019

    2019

  3. [12]

    The pascal visual object classes (voc) challenge

    Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. The pascal visual object classes (voc) challenge. International journal of computer vision, 88:303–338, 2010

    2010

  4. [13]

    Learning federated visual prompt in null space for mri reconstruction

    Chun-Mei Feng, Bangjun Li, Xinxing Xu, Yong Liu, Huazhu Fu, and Wangmeng Zuo. Learning federated visual prompt in null space for mri reconstruction. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8064–8073, 2023

    2023

  5. [14]

    Diverse data augmentation with diffusions for effective test-time prompt tuning

    Chun-Mei Feng, Kai Yu, Yong Liu, Salman Khan, and Wangmeng Zuo. Diverse data augmentation with diffusions for effective test-time prompt tuning. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 2704–2714, 2023

    2023

  6. [15]

    Deep convolutional ranking for multilabel image annotation

    Yunchao Gong, Yangqing Jia, Thomas Leung, Alexander Toshev, and Sergey Ioffe. Deep convolutional ranking for multilabel image annotation. arXiv preprint arXiv:1312.4894, 2013

    2013 arXiv

  7. [16]

    Gener- ative adversarial networks.Communications of the ACM, 63(11):139–144, 2020

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Gener- ative adversarial networks.Communications of the ACM, 63(11):139–144, 2020

    2020

  8. [17]

    I can’t believe there’s no images! learning visual tasks using only language data.arXiv JOURNAL OF LATEX CLASS FILES, VOL

    Sophia Gu, Christopher Clark, and Aniruddha Kembhavi. I can’t believe there’s no images! learning visual tasks using only language data.arXiv JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 15 preprint arXiv:2211.09778, 2022

    2021 arXiv

  9. [18]

    Texts as images in prompt tuning for multi-label image recognition.arXiv preprint arXiv:2211.12739, 2022

    Zixian Guo, Bowen Dong, Zhilong Ji, Jinfeng Bai, Yiwen Guo, and Wangmeng Zuo. Texts as images in prompt tuning for multi-label image recognition.arXiv preprint arXiv:2211.12739, 2022

    2022 arXiv

  10. [19]

    Imagen video: High definition video generation with diffusion models.arXiv preprint arXiv:2210.02303, 2022

    Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P Kingma, Ben Poole, Mohammad Norouzi, David J Fleet, et al. Imagen video: High definition video generation with diffusion models.arXiv preprint arXiv:2210.02303, 2022

    arXiv

  11. [20]

    Learning prompt with distribution-based feature replay for few-shot class-incremental learning.arXiv preprint arXiv:2401.01598, 2024

    Zitong Huang, Ze Chen, Zhixing Chen, Erjin Zhou, Xinxing Xu, Rick Siow Mong Goh, Yong Liu, Wangmeng Zuo, and Chunmei Feng. Learning prompt with distribution-based feature replay for few-shot class-incremental learning.arXiv preprint arXiv:2401.01598, 2024

    2024 arXiv

  12. [21]

    Class concept rep- resentation from contextual texts for training-free multi-label recognition

    Dong Un Kang, Hyunwoo Lee, and Se Young Chun. Class concept rep- resentation from contextual texts for training-free multi-label recognition. OpenReview, 2024

    2024

  13. [22]

    Enhancing clip conceptual embedding through knowl- edge distillation, 2024

    Kuei-Chun Kao. Enhancing clip conceptual embedding through knowl- edge distillation, 2024

    2024

  14. [23]

    Adam: A method for stochastic optimization.arXiv preprint arXiv:1412.6980, 2014

    Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization.arXiv preprint arXiv:1412.6980, 2014

    2014 arXiv

  15. [24]

    Auto-encoding variational bayes

    Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013

    2013 arXiv

  16. [25]

    Openimages: A public dataset for large-scale multi-label and multi-class image classification.Dataset available from https://github

    Ivan Krasin, Tom Duerig, Neil Alldrin, Vittorio Ferrari, Sami Abu-El- Haija, Alina Kuznetsova, Hassan Rom, Jasper Uijlings, Stefan Popov, Andreas Veit, et al. Openimages: A public dataset for large-scale multi-label and multi-class image classification.Dataset available from h...

    2017

  17. [26]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. InComputer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings,...

    2014

  18. [27]

    Compositional visual generation with composable diffusion models

    Nan Liu, Shuang Li, Yilun Du, Antonio Torralba, and Joshua B Tenenbaum. Compositional visual generation with composable diffusion models. InEuropean Conference on Computer Vision, pages 423–439. Springer, 2022

    2022

  19. [28]

    Multi-label image classification via knowledge distillation from weakly-supervised detection

    Yongcheng Liu, Lu Sheng, Jing Shao, Junjie Yan, Shiming Xiang, and Chunhong Pan. Multi-label image classification via knowledge distillation from weakly-supervised detection. InProceedings of the 26th ACM international conference on Multimedia, pages 700–708, 2018

    2018

  20. [29]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017

    2017 arXiv

  21. [30]

    T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models.arXiv preprint arXiv:2302.08453, 2023

    Chong Mou, Xintao Wang, Liangbin Xie, Jian Zhang, Zhongang Qi, Ying Shan, and Xiaohu Qie. T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models.arXiv preprint arXiv:2302.08453, 2023

    2023 arXiv

  22. [31]

    Discriminative region-based multi- label zero-shot learning

    Sanath Narayan, Akshita Gupta, Salman Khan, Fahad Shahbaz Khan, Ling Shao, and Mubarak Shah. Discriminative region-based multi- label zero-shot learning. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 8731–8740, 2021

    2021

  23. [32]

    Glide: Towards photorealistic image generation and editing with text-guided diffusion models.arXiv preprint arXiv:2112.10741, 2021

    Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models.arXiv preprint arXiv:2112.10741, 2021

    2021 arXiv

  24. [33]

    Text-only train- ing for image captioning using noise-injected clip.arXiv preprint arXiv:2211.00575, 2022

    David Nukrai, Ron Mokady, and Amir Globerson. Text-only train- ing for image captioning using noise-injected clip.arXiv preprint arXiv:2211.00575, 2022

    2022 arXiv

  25. [34]

    Sdxl: Improving latent diffusion models for high-resolution image synthesis, 2023

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis, 2023

    2023

  26. [35]

    Semantic-aware representation blending for multi-label image recognition with partial labels

    Tao Pu, Tianshui Chen, Hefeng Wu, and Liang Lin. Semantic-aware representation blending for multi-label image recognition with partial labels. InProceedings of the AAAI conference on artificial intelligence, volume 36, pages 2091–2098, 2022

    2022

  27. [36]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. InInternational conference on machine learning, pag...

    2021

  28. [37]

    Hierarchical text-conditional image generation with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 2022

    2022 arXiv

  29. [38]

    Rethinking prompting strategies for multi-label recognition with partial annotations

    Samyak Rawlekar, Shubhang Bhatnagar, and Narendra Ahuja. Rethinking prompting strategies for multi-label recognition with partial annotations. arXiv preprint arXiv:2409.08381, 2024

    2024 arXiv

  30. [39]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10684–10695, 2022

    2022

  31. [40]

    Dreambooth: Fine tuning text-to- image diffusion models for subject-driven generation.arXiv preprint arXiv:2208.12242, 2022

    Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to- image diffusion models for subject-driven generation.arXiv preprint arXiv:2208.12242, 2022

    2022 arXiv

  32. [41]

    Photorealistic text-to-image diffusion models with deep language understanding.Advances in Neural Information Processing Systems, 35:36479–36494, 2022

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding.Advances in Neural Information ...

    2022

  33. [42]

    Meta-learning for multi-label few-shot classification

    Christian Simon, Piotr Koniusz, and Mehrtash Harandi. Meta-learning for multi-label few-shot classification. InProceedings of the IEEE/CVF winter conference on applications of computer vision, pages 3951–3960, 2022

    2022

  34. [43]

    D2c: Diffusion-decoding models for few-shot conditional generation.Advances in Neural Information Processing Systems, 34:12533–12548, 2021

    Abhishek Sinha, Jiaming Song, Chenlin Meng, and Stefano Ermon. D2c: Diffusion-decoding models for few-shot conditional generation.Advances in Neural Information Processing Systems, 34:12533–12548, 2021

    2021

  35. [44]

    Dualcoop: Fast adaptation to multi-label recognition with limited annotations.arXiv preprint arXiv:2206.09541, 2022

    Ximeng Sun, Ping Hu, and Kate Saenko. Dualcoop: Fast adaptation to multi-label recognition with limited annotations.arXiv preprint arXiv:2206.09541, 2022

    2022 arXiv

  36. [45]

    Vl-adapter: Parameter- efficient transfer learning for vision-and-language tasks

    Yi-Lin Sung, Jaemin Cho, and Mohit Bansal. Vl-adapter: Parameter- efficient transfer learning for vision-and-language tasks. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5227–5237, 2022

    2022

  37. [46]

    Llama: Open and efficient foundation language models.arXiv preprint arXiv:2302.13971, 2023

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie- Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models.arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  38. [47]

    Schwing, and Heng Ji

    Feng Wang, Manling Li, Xudong Lin, Hairong Lv, Alexander G. Schwing, and Heng Ji. Learning to decompose visual features with latent textual prompts, 2022

    2022

  39. [48]

    Cnn-rnn: A unified framework for multi-label image classification

    Jiang Wang, Yi Yang, Junhua Mao, Zhiheng Huang, Chang Huang, and Wei Xu. Cnn-rnn: A unified framework for multi-label image classification. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 2285–2294, 2016

    2016

  40. [49]

    Beyond object proposals: Random crop pooling for multi-label image recognition.IEEE Transactions on Image Processing, 25(12):5678–5688, 2016

    Meng Wang, Changzhi Luo, Richang Hong, Jinhui Tang, and Jiashi Feng. Beyond object proposals: Random crop pooling for multi-label image recognition.IEEE Transactions on Image Processing, 25(12):5678–5688, 2016

    2016

  41. [50]

    Multi-label classification with label graph superimposing

    Ya Wang, Dongliang He, Fu Li, Xiang Long, Zhichao Zhou, Jinwen Ma, and Shilei Wen. Multi-label classification with label graph superimposing. InProceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 12265–12272, 2020

    2020

  42. [51]

    Multi-label image recognition by recurrently discovering attentional regions

    Zhouxia Wang, Tianshui Chen, Guanbin Li, Ruijia Xu, and Liang Lin. Multi-label image recognition by recurrently discovering attentional regions. InProceedings of the IEEE international conference on computer vision, pages 464–472, 2017

    2017

  43. [52]

    Tai++: Text as image for multi-label image classification by co-learning transferable prompt.arXiv preprint arXiv:2405.06926, 2024

    Xiangyu Wu, Qing-Yuan Jiang, Yang Yang, Yi-Feng Wu, Qing-Guo Chen, and Jianfeng Lu. Tai++: Text as image for multi-label image classification by co-learning transferable prompt.arXiv preprint arXiv:2405.06926, 2024

    2024 arXiv

  44. [53]

    Orderless recurrent models for multi-label classification

    Vacit Oguz Yazici, Abel Gonzalez-Garcia, Arnau Ramisa, Bartlomiej Twardowski, and Joost van de Weijer. Orderless recurrent models for multi-label classification. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13440–13449, 2020

    2020

  45. [54]

    Tip-adapter: Training-free adaption of clip for few-shot classification

    Renrui Zhang, Wei Zhang, Rongyao Fang, Peng Gao, Kunchang Li, Jifeng Dai, Yu Qiao, and Hongsheng Li. Tip-adapter: Training-free adaption of clip for few-shot classification. InComputer Vision–ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedin...

    2022

  46. [55]

    Transformer-based dual relation graph for multi-label image recognition

    Jiawei Zhao, Ke Yan, Yifan Zhao, Xiaowei Guo, Feiyue Huang, and Jia Li. Transformer-based dual relation graph for multi-label image recognition. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 163–172, 2021

    2021

  47. [56]

    Con- ditional prompt learning for vision-language models

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Con- ditional prompt learning for vision-language models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16816–16825, 2022

    2022

  48. [57]

    Learning to prompt for vision-language models.International Journal of Computer Vision, 130(9):2337–2348, 2022

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to prompt for vision-language models.International Journal of Computer Vision, 130(9):2337–2348, 2022

    2022

  49. [58]

    Prompt-aligned gradient for prompt tuning.arXiv preprint arXiv:2205.14865, 2022

    Beier Zhu, Yulei Niu, Yucheng Han, Yue Wu, and Hanwang Zhang. Prompt-aligned gradient for prompt tuning.arXiv preprint arXiv:2205.14865, 2022

    2022 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.