Pith. sign in

REVIEW 4 major objections 7 minor 47 references

Unified Framework for Open-World Compositional Zero-shot Learning

T0 review · 4 major / 7 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Single transformer with sparse composition beats CLIP-style models on compositional zero-shot recognition tasks.

desk verdict Genuinely new architecture and strong benchmark numbers, but the TopK selection recall is unmeasured, so the central mechanism's contribution is unverified. read the letter →

arxiv 2412.04083 v1 pith:XUNH5BXY submitted 2024-12-05 cs.CV

Hirunima Jayasekara , Khoi Pham , Nirat Saini , Abhinav Shrivastava This is my paper · ORCID
classification cs.CV
keywords open-worldcompositionalzero-shotlearningvision-languagetransformerTopKselectionsparselinearcompositorattribute-objectcompositionsingle-streammultimodal
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the bottleneck in open-world compositional zero-shot learning is not the size of a vision-language pretraining corpus but the amount of interaction allowed between the image and text modalities. It introduces a single ViT-based transformer that takes image patches and selected attribute/object text embeddings as one sequence, so the two modalities exchange information at every layer. The TopK selection module picks the K most relevant attribute and object words per image, and the Sparse Linear Compositor predicts attributes, objects, and composed pairs with a weight-sharing sparse layer. On MIT-States, C-GQA, and VAW-CZSL, the model reports state-of-the-art AUC over prior OW-CZSL methods, and with 102M parameters and 14M training samples it surpasses CLIP and CSP on C-GQA and VAW-CZSL.

What carries the argument

The argument is carried by two modules. TopK embedding selection performs visual-assisted vocabulary mapping: it uses the image tokens as queries in cross-attention against BERT embeddings of all attributes and objects, sums attention scores, and feeds the top-K attribute and top-K object embeddings to the transformer. The Sparse Linear Compositor (SLC) takes three class tokens: two produce attribute and object logits whose normalized outer product gives a decomposition prediction, while a third, combined with the attribute and object logits, goes through a sparse linear layer where each pair logit is a weighted sum of its attribute and object entries, giving a composition prediction; the final prediction is the scaled sum of both. The sparse layer uses 2(|A|*|O|) parameters instead of (|A|+|O|)(|A|*|O|) for a fully connected layer. Both modules are trainable end-to-end with cross-entropy losses on pairs, on the top attention score, and on attribute and object predictions.

What would settle it

Compute, on the validation split of any of the three datasets, the fraction of images for which the ground-truth attribute and object appear among the TopK module's selected K candidates. If that oracle recall is low while the model still achieves high accuracy, the selection module is not the mechanism the paper claims; a sanity check that replacing the Sparse Linear Compositor with a fully connected head, everything else fixed, yields higher AUC would directly falsify the claimed benefit of the sparse layer.

Watch

Extended reading notes

Core claim

The central discovery is that a single-stream transformer, in which image patches and text embeddings are concatenated and processed together, can outperform much larger dual-stream vision-language models on OW-CZSL when inter-modal interaction is dense. The paper claims that forcing the model to attend jointly across visual and textual tokens throughout all layers, rather than projecting the two modalities into a shared space and taking a dot product, lets the model factorize images into attributes and objects and recombine them into novel pairs. This is supported by the reported numbers: AUC 3.1 versus 2.1 on MIT-States, 1.6 versus 0.87 on C-GQA, and 0.82 versus 0.38 on VAW-CZSL over the re-implemented ViT-B KG-SP baseline, and by beating CLIP and CSP on C-GQA and VAW-CZSL while using about a quarter of CLIP's parameters and about a tenth of its pretraining data. The authors also show the Sparse Linear Compositor improves over a standard fully connected head, for example harmonic mean 7.8 versus 6.1 on C-GQA with 0.8% of the FC parameters.

Load-bearing premise

The load-bearing premise is that selecting exactly K attribute words and K object words by cross-attention score always keeps the words the model needs; if the correct attribute or object is not in the top K, or if gradients cannot flow through the sorting step to the unselected embeddings, the transformer never sees that word and the prediction is structurally impossible to get right.

Editorial extensions

If this is right

  • If correct, open-world compositional zero-shot recognition does not require large-scale vision-language pretraining; a single ViT-B fine-tuned on task data can beat CLIP-based models on some benchmarks.
  • The Sparse Linear Compositor makes the label-space cost scale as O(|A|*|O|) rather than O((|A|+|O|)(|A|*|O|)), so datasets with thousands of attributes and objects become more tractable.
  • TopK selection reduces inference to the selected candidate words, so the approach can serve as a general candidate-generation stage for large open-world label spaces.
  • The hybrid of primitive-wise and joint predictions suggests that both decomposition and composition signals should be trained together, not chosen exclusively.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The authors leave implicit that the TopK module could be trained with a differentiable relaxation such as Gumbel-top-k so gradients reach all candidate embeddings; this is a natural testable extension.
  • The same single-stream recipe may transfer to other compositional recognition tasks, such as color-size-object or action-object, where the label space factorizes into primitives.
  • Since the limitations section notes failures on multiple objects and multiple attributes per object, a concrete extension is to replace the single pair prediction with a set-prediction head over attributes and objects per region.
  • The comparison with CLIP suggests that parameter counts and pretraining-data size are not the dominant factors; a controlled study varying only the depth of inter-modal interaction would sharpen this claim.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The manuscript proposes a unified ViT-B/16-based framework for open-world compositional zero-shot learning (OW-CZSL). The model concatenates image patch embeddings with a fixed BERT-derived attribute/object vocabulary, uses a TopK cross-attention module to select K attribute and K object text embeddings per image, feeds them through a single transformer, and produces attribute, object, and pair predictions via three class tokens and a sparse linear compositor (SLC). Training combines pair, TopK-selection, and auxiliary attribute/object cross-entropy losses. The paper reports state-of-the-art AUC on MIT-States, C-GQA, and VAW-CZSL relative to existing OW-CZSL methods, and reports AUC improvements over CLIP and CSP on C-GQA and VAW-CZSL with fewer parameters and less pre-training data. Ablations on MIT-States examine the TopK width and the sparse compositor.

Significance. If the reported numbers are reproducible, the contributions are meaningful: a single-stream transformer is shown to be competitive with much larger CLIP-based models on two of three OW-CZSL benchmarks, the parameter count of the compositor is reduced by orders of magnitude relative to a fully connected layer, and the TopK module offers a way to limit the language input without exhaustive pair search. The paper contains no circular reasoning: the claims are empirical comparisons on external benchmarks, and the ablations provide some support for the design choices. The code link and the explicit use of public benchmark protocols are also strengths. However, the absence of multiple seeds/error bars and of several centrally tuned hyperparameter values means the quantitative claims should be treated as preliminary until reproduced.

major comments (4)
  1. [§3.4 and §3.7] The reported gains are attributed to the TopK selection module, but no selection-recall metric is reported. The module's only training signal is a cross-entropy loss on the single highest attention score for attributes and objects, which optimizes rank-1 correctness, not the probability that the ground-truth primitive is within the top K; the K-ablation in Table 3 changes K only on MIT-States and does not report recall at K. Because the K=3 text embeddings per modality are the only language input to the transformer, a low recall would mean the final pair prediction is often made without the correct text token, so the mechanism behind the reported state-of-the-art numbers would remain unexplained. Please report attribute/object recall@K on the test splits, ideally separated by seen and unseen compositions, and if possible measure how often the ground-truth text token is absent from the selected K for correct versus incorrect pair predictions.
  2. [§4.1.2, §3.6, §3.7] Hyperparameters central to the method are named but their values are missing. K is stated only through the MIT-States ablation and not for C-GQA/VAW-CZSL; the scale factor η in Eq. (14), the loss weights α1–α3 in Eq. (17), and the feasibility threshold in §3.6 are each described as empirically set, but no values are given in the main text or in the arXiv version. Without these values the experiments cannot be reproduced, and because the feasibility mask directly filters the output space, its threshold materially affects all reported AUC numbers. Please provide a complete hyperparameter table for all datasets, including K, η, αi, the feasibility threshold, and any layer-wise learning-rate decay settings.
  3. [Table 2 and §4.3] The central comparisons are reported as single numbers without variance. The main AUC improvements over the re-implemented KG-SP ViT baseline are 1.0, 0.73, and 0.44 on MIT-States, C-GQA, and VAW-CZSL respectively; with one seed and no error bars these differences cannot be distinguished from training noise, especially for the smaller VAW-CZSL gain. The ablations in Table 3 are also on MIT-States only with a fixed random seed. Please report mean ± standard deviation over at least three seeds for the main table and for the TopK ablation, or explicitly state if compute constraints make this infeasible.
  4. [§3.4, Eq. (4)–(5)] As written, the attention computation is dimensionally inconsistent. In Eq. (2), \tilde v is a matrix of N+3 tokens by H (with H = P^2·C), while uvocab[0:|A|] is |A| by H, so the product in Eq. (4) is undefined unless one of the two is transposed or pooled; Eq. (5) also states a sum over the P^2·C dimension of A_a, but A_a's first dimension would be the number of image tokens if the transposed product is intended. Please clarify the exact shapes and reduction used to produce the attention score vector \tilde A_a, since this is the core of the proposed selection mechanism.
minor comments (7)
  1. [Abstract] The acronym 'LLVM' for 'Large Vision Language Models' should be 'LVLM' or the phrase should be written out as 'large vision-language models'; LLVM conventionally refers to compiler infrastructure.
  2. [§4.1.2] The sentence 'We fine-tune the proposed model for 20 epochs for VAW-CZSL and 10 epochs for other three datasets' is inaccurate because only three datasets are used in total; it should say 'other two datasets' (MIT-States and C-GQA).
  3. [§4.2.1] The baseline name 'KS-SP' is a typo and should be 'KG-SP'.
  4. [§2] The subsection numbering '2.0.1', '2.0.2', etc. should be regular numbered subsections such as 2.1, 2.2, 2.3, and 2.4.
  5. [§3.7, Eq. (17)] The loss notation is inconsistent: the first term is written as 'L_Pair' in text and 'LP air' in the displayed equation; please standardize the notation for all loss terms.
  6. [Table 5] The header 'Total # Par.' should be expanded to 'Total # Params', and the meaning of the '14M' value for Ours should be stated explicitly as the number of training images used by the proposed method.
  7. [§4.3.2] The phrase 'which can be contributed to overfitting' should be 'which can be attributed to overfitting'.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the central claims are empirical benchmark comparisons, and the same-author dataset citations are not load-bearing reductions.

full rationale

The paper's central claim, state-of-the-art OW-CZSL results in Tables 2 and 5, is an empirical performance comparison against baseline models on three benchmark datasets. None of the reported metrics (seen accuracy, unseen accuracy, HM, AUC) is derived from the model's own inputs by construction. The TopK selection module in Sec. 3.4 is a learned gating mechanism trained with a cross-entropy loss on the highest attention score, and the final pair prediction is produced by the Sparse Linear Compositor in Sec. 3.5.1; the evaluation numbers are measured on held-out splits using the established Purushwalkam et al. protocol, not recomputed from training labels. The paper includes self-citations to prior work by the same authors, notably the VAW-CZSL benchmark [37] and related attribute prediction work [32, 33], but these citations supply a concrete dataset artifact with fixed splits and established baselines rather than assuming the target result. The skeptical concern that TopK selection may exclude the correct attribute or object for some test images is a legitimate correctness-risk question about whether the reported accuracy is attributable to the selection mechanism, but it is not circularity: the paper reports end-task accuracy and an ablation that removes TopK, and no equation reduces the final prediction to the TopK training objective. The limitation discussion in Sec. 4.5.1 candidly notes failures with multiple objects and multiple attributes, which is an honest statement of scope rather than a circular step. No specific reduction of the kind required by the circularity rules could be exhibited, so the appropriate finding is no significant circularity; the score of 1 reflects only the minor presence of same-author citations in the benchmark lineage, which is not load-bearing.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The method relies on standard empirical assumptions about transfer learning and feasibility filtering, and on several hand-set hyperparameters (K, eta, alpha weights, feasibility threshold) that are tuned on the target datasets and not reported in the main text. No new physical entities are introduced.

free parameters (4)
  • K = 3
    Number of attribute and object text embeddings selected by TopK; chosen empirically after ablations K=1, 3, 5, 10 in Section 4.3.1, with higher K degrading accuracy.
  • eta scale factor = not reported in main text
    Scale factor combining decomposition and composition predictions in Eq. 14; the value is said to be empirically set but is omitted from the main text.
  • alpha loss weights = not reported in main text
    Weights for TopK, attribute, and object losses in Eq. 17; described as empirically computed, with values deferred to the supplementary.
  • feasibility threshold = not reported
    Threshold for the GloVe and ConceptNet feasibility mask in Section 3.6; authors state it is set by empirically adjusting the value, and it directly determines which candidate pairs survive.
assumptions (4)
  • domain assumption Fixed BERT embeddings for attribute and object names provide sufficient semantics for the open-world vocabulary.
    Section 3.3.2 builds the vocabulary from BERT embeddings that are not updated during training; if BERT represents visual attributes poorly, the text signal fed to the transformer is weak.
  • domain assumption Weights pretrained on ImageNet-21K for a vision transformer can be reused to process a mixed image-and-text token sequence.
    Section 3.5 initializes the transformer from ViT-B/16 weights rather than BERT and adds only small type embeddings; the paper does not validate whether this transfer works for text tokens.
  • domain assumption GloVe plus ConceptNet cosine similarity is a reliable proxy for attribute-object feasibility.
    Section 3.6 uses these scores to build the binary mask fpair with an empirical threshold; any valid pair below the threshold is permanently removed from the output space, which can suppress unseen pairs.
  • ad hoc to paper Hard TopK selection can be trained end-to-end with supervision on only the highest attention score.
    Section 3.4 selects top-K tokens by sorting non-differentiable indices, and Section 3.7 applies cross-entropy only to the highest attention scores; no gradient estimator for the discrete selection is described.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Unified Framework for Open-World Compositional Zero-shot Learning." pith.science (2026). https://pith.science/paper/XUNH5BXY

@misc{pith2026241204083,
  author       = {Pith},
  title        = {Pith review of: Unified Framework for Open-World Compositional Zero-shot Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XUNH5BXY}},
  note         = {Machine review of arXiv:2412.04083}
}
read the original abstract

Open-World Compositional Zero-Shot Learning (OW-CZSL) addresses the challenge of recognizing novel compositions of known primitives and entities. Even though prior works utilize language knowledge for recognition, such approaches exhibit limited interactions between language-image modalities. Our approach primarily focuses on enhancing the inter-modality interactions through fostering richer interactions between image and textual data. Additionally, we introduce a novel module aimed at alleviating the computational burden associated with exhaustive exploration of all possible compositions during the inference stage. While previous methods exclusively learn compositions jointly or independently, we introduce an advanced hybrid procedure that leverages both learning mechanisms to generate final predictions. Our proposed model, achieves state-of-the-art in OW-CZSL in three datasets, while surpassing Large Vision Language Models (LLVM) in two datasets.

Figures

Figures reproduced from arXiv: 2412.04083 by the authors.

Figure 1
Figure 1. The overall architecture of the proposed method. Input embeddings to the transformer encoder are formed by concatenating [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

47 extracted references · 33 canonical work pages

  1. [1]

    A causal view of compositional zero-shot recognition

    Yuval Atzmon, Felix Kreuk, Uri Shalit, and Gal Chechik. A causal view of compositional zero-shot recognition. Ad- vances in Neural Information Processing Systems, 33:1462– 1473, 2020. 6

    2020

  2. [2]

    Vlmo: Unified vision-language pre-training with mixture-of-modality-experts

    Hangbo Bao, Wenhui Wang, Li Dong, Qiang Liu, Owais Khan Mohammed, Kriti Aggarwal, Subhojit Som, Songhao Piao, and Furu Wei. Vlmo: Unified vision-language pre-training with mixture-of-modality-experts. Advances in Neural Information Processing Systems , 35:32897–32912,

    work page

  3. [3]

    Class-based n-gram models of natural language

    PF Brown, PV DeSouza, RL Mercer, VJ Della Pietra, and JC Lai. Class-based n-gram models of natural language. Com- put. Linguist, (1950), 1992. 3

    work page 1950

  4. [4]

    Multimodal pretraining unmasked: A meta-analysis and a unified framework of vision-and- language berts

    Emanuele Bugliarello, Ryan Cotterell, Naoaki Okazaki, and Desmond Elliott. Multimodal pretraining unmasked: A meta-analysis and a unified framework of vision-and- language berts. Transactions of the Association for Com- putational Linguistics, 9:978–994, 2021. 3

    work page 2021

  5. [5]

    Uniter: Learning universal image-text representations

    Yen-Chun Chen, Linjie Li, Licheng Yu, Ahmed El Kholy, Faisal Ahmed, Zhe Gan, Yu Cheng, and Jingjing Liu. Uniter: Learning universal image-text representations. 2019. 3

    work page 2019

  6. [6]

    On the difference of bert-style and clip-style text encoders

    Zhihong Chen, Guiming Hardy Chen, Shizhe Diao, Xiang Wan, and Benyou Wang. On the difference of bert-style and clip-style text encoders. arXiv preprint arXiv:2306.03678 ,

    arXiv

  7. [7]

    An image is worth 16x16 words: Trans- formers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, et al. An image is worth 16x16 words: Trans- formers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020. 3, 4

    arXiv 2010

  8. [8]

    Learning at- tention as disentangler for compositional zero-shot learning

    Shaozhe Hao, Kai Han, and Kwan-Yee K Wong. Learning at- tention as disentangler for compositional zero-shot learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15315–15324, 2023. 3, 6

    work page 2023

Show all 47 references
  1. [9]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 6

    2016

  2. [10]

    Parts of recog- nition

    Donald D Hoffman and Whitman A Richards. Parts of recog- nition. Cognition, 18(1-3):65–96, 1984. 2

    1984

  3. [11]

    Gqa: A new dataset for real-world visual reasoning and compositional question answering

    Drew A Hudson and Christopher D Manning. Gqa: A new dataset for real-world visual reasoning and compositional question answering. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 6700–6709, 2019. 5

    2019

  4. [12]

    Dis- covering states and transformations in image collections

    Phillip Isola, Joseph J Lim, and Edward H Adelson. Dis- covering states and transformations in image collections. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1383–1391, 2015. 5, 7

    2015

  5. [13]

    Kg-sp: Knowledge guided simple primitives for open world compositional zero-shot learning

    Shyamgopal Karthik, Massimiliano Mancini, and Zeynep Akata. Kg-sp: Knowledge guided simple primitives for open world compositional zero-shot learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9336–9345, 2022. 1, 2, 3, 5, 6

    2022

  6. [14]

    Bert: Pre-training of deep bidirectional trans- formers for language understanding

    Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. Bert: Pre-training of deep bidirectional trans- formers for language understanding. In Proceedings of naacL-HLT, volume 1, page 2, 2019. 4

    2019

  7. [15]

    Hierarchical visual primitive experts for composi- tional zero-shot learning

    Hanjae Kim, Jiyoung Lee, Seongheon Park, and Kwanghoon Sohn. Hierarchical visual primitive experts for composi- tional zero-shot learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5675– 5685, 2023. 1, 2, 6

    2023

  8. [16]

    Vilt: Vision- and-language transformer without convolution or region su- pervision

    Wonjae Kim, Bokyung Son, and Ildoo Kim. Vilt: Vision- and-language transformer without convolution or region su- pervision. In International Conference on Machine Learn- ing, pages 5583–5594. PMLR, 2021. 2, 3, 4

    2021

  9. [17]

    Visualbert: A simple and perfor- mant baseline for vision and language

    Liunian Harold Li, Mark Yatskar, Da Yin, Cho-Jui Hsieh, and Kai-Wei Chang. Visualbert: A simple and perfor- mant baseline for vision and language. arXiv preprint arXiv:1908.03557, 2019. 3

    1908 arXiv

  10. [18]

    Grounded language-image pre-training

    Liunian Harold Li, Pengchuan Zhang, Haotian Zhang, Jian- wei Yang, Chunyuan Li, Yiwu Zhong, Lijuan Wang, Lu Yuan, Lei Zhang, Jenq-Neng Hwang, et al. Grounded language-image pre-training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, page...

    2022

  11. [19]

    Distilled re- verse attention network for open-world compositional zero- shot learning

    Yun Li, Zhe Liu, Saurav Jha, and Lina Yao. Distilled re- verse attention network for open-world compositional zero- shot learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1782–1791, 2023. 2, 6

    2023

  12. [20]

    Symme- try and group in attribute-object compositions

    Yong-Lu Li, Yue Xu, Xiaohan Mao, and Cewu Lu. Symme- try and group in attribute-object compositions. In Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11316–11325, 2020. 2, 6

    2020

  13. [21]

    Simple primi- tives with feasibility-and contextuality-dependence for open- world compositional zero-shot learning

    Zhe Liu, Yun Li, Lina Yao, Xiaojun Chang, Wei Fang, Xi- aojun Wu, and Abdulmotaleb El Saddik. Simple primi- tives with feasibility-and contextuality-dependence for open- world compositional zero-shot learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 202...

    2023

  14. [22]

    Fixing weight decay reg- ularization in adam

    Ilya Loshchilov and Frank Hutter. Fixing weight decay reg- ularization in adam. 2018. 6

    2018

  15. [23]

    Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks

    Jiasen Lu, Dhruv Batra, Devi Parikh, and Stefan Lee. Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks. Advances in neural information processing systems, 32, 2019. 3

    2019

  16. [24]

    Open world compositional zero- shot learning

    Massimiliano Mancini, Muhammad Ferjad Naeem, Yongqin Xian, and Zeynep Akata. Open world compositional zero- shot learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 5222– 5230, 2021. 1, 3, 5, 6

    2021

  17. [25]

    From red wine to red tomato: Composition with context

    Ishan Misra, Abhinav Gupta, and Martial Hebert. From red wine to red tomato: Composition with context. In Proceed- ings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1792–1801, 2017. 2, 3, 6

    2017

  18. [26]

    Cross-stitch networks for multi-task learning

    Ishan Misra, Abhinav Shrivastava, Abhinav Gupta, and Mar- tial Hebert. Cross-stitch networks for multi-task learning. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3994–4003, 2016. 3

    2016

  19. [27]

    Learning graph embeddings for compositional zero-shot learning

    Muhammad Ferjad Naeem, Yongqin Xian, Federico Tombari, and Zeynep Akata. Learning graph embeddings for compositional zero-shot learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 953–962, 2021. 1, 6

    2021

  20. [28]

    Attributes as op- erators: factorizing unseen attribute-object compositions

    Tushar Nagarajan and Kristen Grauman. Attributes as op- erators: factorizing unseen attribute-object compositions. In Proceedings of the European Conference on Computer Vi- sion (ECCV), pages 169–185, 2018. 2

    2018

  21. [29]

    Learning to compose soft prompts for compositional zero-shot learning

    Nihal V Nayak, Peilin Yu, and Stephen H Bach. Learning to compose soft prompts for compositional zero-shot learning. arXiv preprint arXiv:2204.03574, 2022. 1, 2, 3, 5, 7, 8

    2022 arXiv

  22. [30]

    Movie: Revisiting modulated convolutions for visual count- ing and beyond

    Duy-Kien Nguyen, Vedanuj Goswami, and Xinlei Chen. Movie: Revisiting modulated convolutions for visual count- ing and beyond. arXiv preprint arXiv:2004.11883, 2020. 2

    2004 arXiv

  23. [31]

    Glove: Global vectors for word representation

    Jeffrey Pennington, Richard Socher, and Christopher D Man- ning. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP) , pages 1532–1543,

    2014

  24. [32]

    Learning to predict visual attributes in the wild

    Khoi Pham, Kushal Kafle, Zhe Lin, Zhihong Ding, Scott Cohen, Quan Tran, and Abhinav Shrivastava. Learning to predict visual attributes in the wild. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 13018–13028, 2021. 5

    2021

  25. [33]

    Improving closed and open-vocabulary attribute prediction using transformers

    Khoi Pham, Kushal Kafle, Zhe Lin, Zhihong Ding, Scott Co- hen, Quan Tran, and Abhinav Shrivastava. Improving closed and open-vocabulary attribute prediction using transformers. In European Conference on Computer Vision , pages 201–

  26. [34]

    Task-driven modular networks for zero-shot compositional learning

    Senthil Purushwalkam, Maximilian Nickel, Abhinav Gupta, and Marc’Aurelio Ranzato. Task-driven modular networks for zero-shot compositional learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 3593–3602, 2019. 5, 6

    2019

  27. [35]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  28. [36]

    Zero-shot text-to-image generation

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In International Confer- ence on Machine Learning, pages 8821–8831. PMLR, 2021. 3

    2021

  29. [37]

    Disentan- gling visual embeddings for attributes and objects

    Nirat Saini, Khoi Pham, and Abhinav Shrivastava. Disentan- gling visual embeddings for attributes and objects. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 13658–13667, June

  30. [38]

    Flava: A foundational language and vision alignment model

    Amanpreet Singh, Ronghang Hu, Vedanuj Goswami, Guil- laume Couairon, Wojciech Galuba, Marcus Rohrbach, and Douwe Kiela. Flava: A foundational language and vision alignment model. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pages 1563...

    2022

  31. [39]

    Concept- net 5.5: An open multilingual graph of general knowledge

    Robyn Speer, Joshua Chin, and Catherine Havasi. Concept- net 5.5: An open multilingual graph of general knowledge. In Proceedings of the AAAI conference on artificial intelli- gence, volume 31, 2017. 3, 5

    2017

  32. [40]

    Lxmert: Learning cross- modality encoder representations from transformers

    Hao Tan and Mohit Bansal. Lxmert: Learning cross- modality encoder representations from transformers. arXiv preprint arXiv:1908.07490, 2019. 3

    1908 arXiv

  33. [41]

    Learning conditional attributes for compositional zero-shot learning

    Qingsheng Wang, Lingqiao Liu, Chenchen Jing, Hao Chen, Guoqiang Liang, Peng Wang, and Chunhua Shen. Learning conditional attributes for compositional zero-shot learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11197–11206, 2023. 1, 2

    2023

  34. [42]

    Relation-aware compositional zero-shot learn- ing for attribute-object pair recognition

    Ziwei Xu, Guangzhi Wang, Yongkang Wong, and Mohan S Kankanhalli. Relation-aware compositional zero-shot learn- ing for attribute-object pair recognition. IEEE Transactions on Multimedia, 24:3652–3664, 2021. 1, 2

    2021

  35. [43]

    Unified contrastive learning in image-text-label space

    Jianwei Yang, Chunyuan Li, Pengchuan Zhang, Bin Xiao, Ce Liu, Lu Yuan, and Jianfeng Gao. Unified contrastive learning in image-text-label space. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 19163–19173, 2022. 3

    2022

  36. [44]

    Learning unseen concepts via hierarchi- cal decomposition and composition

    Muli Yang, Cheng Deng, Junchi Yan, Xianglong Liu, and Dacheng Tao. Learning unseen concepts via hierarchi- cal decomposition and composition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10248–10256, 2020. 1, 2

    2020

  37. [45]

    Coca: Contrastive captioners are image-text foundation models

    Jiahui Yu, Zirui Wang, Vijay Vasudevan, Legg Yeung, Mo- jtaba Seyedhosseini, and Yonghui Wu. Coca: Contrastive captioners are image-text foundation models. arXiv preprint arXiv:2205.01917, 2022. 3

    2022 arXiv

  38. [46]

    Revisiting few-sample bert fine- tuning

    Tianyi Zhang, Felix Wu, Arzoo Katiyar, Kilian Q Wein- berger, and Yoav Artzi. Revisiting few-sample bert fine- tuning. In International Conference on Learning Represen- tations, 2020. 6

    2020

  39. [47]

    Learning to prompt for vision-language models

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to prompt for vision-language models. In- ternational Journal of Computer Vision, 130(9):2337–2348,

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.