Pith. sign in

REVIEW 3 major objections 6 minor 78 references

DART: Dual Adaptive Refinement Transfer for Open-Vocabulary Multi-Label Recognition

T0 review · 3 major / 6 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read DART claims that a frozen CLIP backbone, fitted with adaptive patch refinement and an LLM-mined class graph transfer module, outperforms prior open-vocabulary multi-label recognition methods on NUS-WIDE, MS-COCO, and Open Images.

desk verdict DART reports strong gains on OV-MLR, but the main source of those gains—the LLM-built class graph—is exactly where benchmark leakage could hide, and the paper doesn't audit it. read the letter →

arxiv 2508.05585 v1 pith:LBWSWON5 submitted 2025-08-07 cs.CV

Haijing Liu , Tao Pu , Hefeng Wu , Keze Wang , Liang Lin This is my paper
classification cs.CV
keywords open-vocabularymulti-labelrecognitionvision-languagemodelCLIPweaklysupervisedpatchselectiongraphattentionnetworkLLM-minedclassrelationshipzero-shotlearningimage
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Open-vocabulary multi-label recognition must name every object in an image, including categories never seen in training, and it typically breaks down in two places: the vision encoder cannot localize objects precisely, and nothing connects a novel class to classes the model already knows. This paper claims both failures are repairable on top of a frozen CLIP. An Adaptive Refinement Module, trained by a weakly supervised patch-selecting loss, sharpens patch-level localization using only image-level labels; an Adaptive Transfer Module runs graph attention over an LLM-mined class-relationship graph so relational context flows from seen classes to unseen ones. If the claim holds, state-of-the-art open-vocabulary tagging does not require retraining a large backbone, and a system with only a few million trainable parameters can surpass both earlier specialist methods and far larger generative multimodal models. The paper reports new best results on NUS-WIDE, MS-COCO, and Open Images, with its clearest gain on NUS-WIDE zero-shot mAP (43.9 versus 37.6 for the previous best method).

What carries the argument

The Adaptive Refinement Module (ARM) is parasitic on later transformer blocks, producing input-adaptive residuals $\Delta\tilde{x}$ via LoRA-adjusted attention, depthwise convolution, and cross-attention gating, added to the frozen patch features. Its training signal is the Weakly Supervised Patch-Selecting (WPS) loss, a tractable EM-style objective: a softmax over patch-class scores assigns each patch a responsibility $\hat{z}_{i,c}$; positive classes reinforce confident patches, hard-negative mining suppresses the strongest patches of absent classes. The Adaptive Transfer Module (ATM) is a two-stage GATv2 message passer over the LLM-mined Class Relationship Graph (CRG), each class's top-$N

What would settle it

Rebuild the Class Relationship Graph without access to the benchmarks' own label statistics — e.g., prompt the LLM with paraphrased or synonym-renamed class concepts instead of the literal NUS-WIDE/MS-COCO/Open Images class lists, or derive edges from a neutral source such as WordNet — and re-measure NUS-WIDE zero-shot mAP. If the reported margin over MKT (43.9 vs 37.6) shrinks or disappears, part of the 'unseen class' gain is leakage of benchmark co-occurrence knowledge rather than transfer from general world knowledge.

Watch

Extended reading notes

Core claim

DART claims a frozen CLIP backbone fails at open-vocabulary multi-label recognition because patch activations are noisy and no relational knowledge links seen to unseen classes; jointly trained modules fix both. An Adaptive Refinement Module (ARM) refines frozen patch features under a Weakly Supervised Patch-Selecting loss that, framed as expectation-maximization on a bag of patches, picks class-relevant patches from image-level labels. An Adaptive Transfer Module (ATM) runs GATv2 message passing over an LLM-mined Class Relationship Graph, on text embeddings, then on fused features, giving unseen classes context from related classes. Reported results are state-of-the-art on three benchmarks.

Load-bearing premise

The load-bearing premise is that the LLM-mined class graph encodes general world knowledge rather than co-occurrence statistics of the three benchmarks, since it is built by prompting the LLM with those datasets' exact class lists and no contamination check is reported; a secondary fragility is that the WPS patch 'responsibilities' are a softmax approximation the paper itself flags rather than a derived posterior.

Editorial extensions

If this is right

  • The state-of-the-art numbers come with a small budget: about 6.5M trainable parameters on a frozen ViT-B/16, with per-batch training time of 106.8 ms versus 172.5 ms for the prior method MKT.
  • Because the class graph is mined from an LLM rather than from training-set co-occurrence, inter-class transfer exists for classes never seen in training; swapping the LLM graph for a random graph drops NUS-WIDE zero-shot mAP from 43.9 to 33.4 (a 23.9% relative loss).
  • The WPS positive term is effectively a SigLIP-style sigmoid loss with dynamically computed per-patch responsibilities, so the ARM+WPS pair is a general mechanism for weakly supervised localization with frozen VLP features, beyond multi-label recognition.
  • On a NUS-WIDE subset, DART with roughly 100M total parameters outperforms generative multimodal LLMs of 7B-11B parameters on zero-shot and generalized zero-shot F1, with much higher recall and far less out-of-vocabulary prediction.
  • Ablations attribute mAP gains to ARM+WPS localization and F1 gains to ATM relational transfer, so jointly optimizing refinement and transfer — the paper's stated novelty — is what produces the headline results.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The CRG is mined with the exact class lists of the evaluation datasets and is not released; if the LLM's training data includes co-occurrence statistics of NUS-WIDE, MS-COCO, or Open Images, part of the 'unseen class' gain could be benchmark leakage rather than transferable world knowledge. The clean test is to rebuild the graph from paraphrased class names or a neutral knowledge source like WordN
  • The paper itself flags the WPS E-step as a softmax simplification rather than a Bayes-derived posterior, so the 'theoretically grounded' claim is provisional; comparing against a properly iterated EM or variational responsibility estimate would show whether the approximation is what drives the mAP gain.
  • If the claimed positive-feedback loop is real, the modules should be mutually load-bearing: lowering graph quality (random versus LLM edges) should change how much the ARM helps, and vice versa. A two-way ablation of graph source by ARM on/off would expose whether the loop is genuinely synergistic or dominated by one direction.
  • The machinery is not obviously tied to multi-label recognition: the same ARM+WPS patch selection and ATM class transfer should transfer to frozen encoders other than CLIP and to open-vocabulary detection or segmentation, where patch-level localization and cross-class context are also the bottleneck.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes DART, a framework for open-vocabulary multi-label recognition (OV-MLR) built on a frozen CLIP ViT-B/16 backbone. Two trainable modules are added: an Adaptive Refinement Module (ARM) that refines patch features under image-level supervision and is trained with a Weakly Supervised Patch Selecting (WPS) loss, and an Adaptive Transfer Module (ATM) that propagates information over a Class Relationship Graph (CRG) mined from an LLM using GATv2 layers. The training objective is Eq. (1): a ranking classification loss plus WPS and an L1 penalty on the refinement residual. Experiments on NUS-WIDE, MS-COCO, and Open Images report state-of-the-art ZSL/GZSL results, with ablations for each module, for loss components, for the CRG source, and for several hyperparameters, together with qualitative localization and relation visualizations.

Significance. If the reported results are correct, DART is a practical and effective recipe for OV-MLR: it obtains consistent improvements over prior methods across three benchmarks, e.g., Table 1 ZSL mAP 43.9 vs. MKT's 37.6 and ZSL F1@3 36.1 vs. 34.1, plus gains on Open Images (Table 2) and MS-COCO (Table 3). The paper's strengths are the per-component ablations, the hyperparameter studies (Figures 4 and 7), and the explicit qualitative evidence for better localization. The main weakness is that the key novelty, the LLM-mined CRG, is not released and is vulnerable to a contamination concern that is not addressed. The WPS loss also is described as 'theoretically grounded' although the derivation itself concedes that the posterior approximation is a simplification. The empirical core is plausible but needs a reproducibility and contamination audit before the central open-vocabulary claim is accepted.

major comments (3)
  1. [Sec. 3.4, App. C.4, Table 6] The central claim of open-vocabulary transfer rests on the LLM-mined CRG, but the construction protocol leaves a concrete contamination route. App. C.4 gives gpt-4o-latest the exact class lists of the evaluation datasets and explicitly lists 'Co-occurrence' as an allowed relationship type. Because the ATM propagates evidence from seen to unseen class nodes over these edges, LLM knowledge that mirrors benchmark co-occurrence can act as a dataset-specific shortcut. Table 6 shows the graph choice is decisive (random graph ZSL mAP 33.4 vs. LLM graph 43.9), so the improvement is concentrated exactly where contamination is possible. The paper's Fig. 1b contrast with dataset co-occurrence statistics is not established, since the prompt does not exclude co-occurrence knowledge. I request a contamination audit: compare against a graph built from actual training-set co-occurrence, ablate the co-oc
  2. [Sec. 3.3, Eq. (4), App. C.1.3] The WPS loss is introduced as 'theoretically grounded,' but the derivation itself acknowledges that the softmax responsibility is 'a common simplification for tractability, rather than a direct derivation from Bayes' rule under MIL assumptions.' App. C.1.3 shows the Bayes posterior is intractable and then substitutes a softmax without a variational bound or error analysis. In addition, the final loss in Eq. (4) drops the second BCE term for positive patches after substituting the responsibilities, which is a heuristic choice. The empirical ablations show the loss works, but the paper should describe it as a designed instance-weighted contrastive/MIL objective and remove or qualify the 'theoretically grounded' wording in the contributions and in Sec. 3.3.
  3. [Tabs. 1-4, App. A] The empirical comparisons are reported as single numbers with no error bars, confidence intervals, or multiple seeds, and no code or CRG is released. Given the many hyperparameters tuned on these benchmarks (gamma_wps, gamma_penalty, lambda schedule, K, N, LoRA rank, batch size), it is difficult to assess whether headline gaps such as ZSL F1@3 36.1 vs. 34.1 in Table 1 are significant or are partly due to hyperparameter selection. The CRG construction is particularly unrepeatable: the exact prompt template, aggregated scores, and edge lists are not provided. Please release the code and graph, and report at least 3 seeds with mean and std for the main tables.
minor comments (6)
  1. [Tables 4 and 7] The parameter counts are inconsistent: Sec. 4.2 says 'only 123M parameters,' Table 4 says '100M,' and Table 7 reports 92.7M total. Please reconcile.
  2. [Abstract / Sec. 5] The 'first framework, to our knowledge' claim is stronger than the related-work survey supports, given existing graph-based and LLM-knowledge methods in adjacent tasks. Please broaden the comparison or soften the claim.
  3. [Sec. 2] Typo: 'they struggles' should be 'they struggle.' Also references [5] and [6] are duplicates.
  4. [Table 6] The column header reads 'GSL' instead of 'GZSL.'
  5. [App. A] The reference to 'WPS contrastive loss (Eq. 3)' is incorrect; the WPS loss is Eq. (4) in the main text.
  6. [Fig. 4] The x-axis labels omit the gamma symbols; label them γ_penalty and γ_wps for consistency with Eq. (1).

Circularity Check

0 steps flagged · score 0.0 of 10

No in-paper circularity: the central SOTA claims are externally benchmarked; the WPS self-referentiality is an acknowledged training approximation, and the CRG contamination risk is an external-validity concern, not an equation-level circularity.

full rationale

Walking DART's derivation chain, the final predictions are produced by a frozen CLIP backbone plus trainable ARM/WPS, ATM/GAT, and a ranking loss supervised by image-level labels, then evaluated on external benchmarks (NUS-WIDE, MS-COCO, Open Images). No load-bearing step reduces to its own inputs by construction: the CRG is constructed offline (Sec. 3.4, App. C.4) rather than fitted to the training labels; the GAT message passing in Eq. (5) is standard and benchmark-validated; and the LLM-derived graph is not itself a prediction of the paper, so its contamination risk is a leakage/validity concern, not circularity. The WPS loss's positive responsibilities are softmax functions of the model's own patch scores (Sec. 3.3: 'we approximate this responsibility using a Softmax function over the predicted patch scores'), and the paper explicitly labels this a 'common simplification for tractability, rather than a direct derivation from Bayes' rule' (App. C.1.3). That is a self-referential training objective, but it is auxiliary to a label-driven ranking loss and is presented as an acknowledged approximation, not as an independent prediction derived from first principles. Self-citations (e.g., [37]) are contextual and not load-bearing; no uniqueness theorem or fitted parameter is renamed as a prediction. Therefore, the manuscript contains no demonstrated circular step; the flagged CRG contamination risk and the WPS simplification are empirical/validity caveats deserving audit, but they do not make the paper's central derivation circular.

Assumptions & free parameters 9 free parameters · 6 assumptions · 1 invented entities

The central empirical gains rest on a handful of assumptions: the MIL label-to-patch link, the softmax approximation for patch responsibilities, the quality and non-contamination of the LLM-mined graph, and the preservation of CLIP's open-vocabulary knowledge under residual refinement. The WPS loss depends on several hand-chosen or ablated hyperparameters (gamma_wps, gamma_penalty, lambda schedule, K, N, batch size, LoRA rank), and the CRG itself is an unreleased artifact generated by a stochastic LLM. These are not fatal, but they bound the claim: the paper demonstrates the method works under its chosen settings, it does not yet demonstrate a parameter-free or closed-form result.

free parameters (9)
  • gamma_wps = 5
    WPS loss weight; optimal in Fig 4d ablation; balances localization (mAP) and F1.
  • gamma_penalty = not stated numerically; ablated 0.1 to 10
    L1 penalty weight on ARM residual; prevents catastrophic forgetting (Fig 4c).
  • lambda schedule = scheduled annealing, exact rule not given
    Interpolates between original VLP responsibilities and model responsibilities in WPS; Appendix D.2.
  • K = 16
    Number of hard negatives and top-K patch aggregates; ablated in Fig 4b and Fig 7c.
  • CRG neighbor count N = 8 for NUS-WIDE/MS-COCO, 12 for Open Images
    CRG connectivity; ablated in Fig 7b.
  • LoRA rank r and alpha = r=32, alpha=128
    ARM capacity; chosen via preliminary experiments (Appendix A).
  • batch size = 256
    Ablated in Fig 7a; optimal in tested range.
  • temperature tau = fixed to CLIP temperature
    Score scaling in WPS; main text says learnable, appendix says fixed.
  • GAT layers = 2 layers per ATM stage; heads not stated
    ATM depth; implementation detail.
assumptions (6)
  • domain assumption Image-level positive label implies at least one patch corresponds to class; negative label implies no patch corresponds (standard MIL assumption).
    Sec 3.3 and Appendix C.1.2 define the label-to-patch link used to derive WPS.
  • ad hoc to paper Softmax over patch scores approximates the posterior P(z_i,c=1|y_c=1) in the E-step.
    Appendix C.1.3 admits this is a simplification rather than a derivation from Bayes' rule.
  • domain assumption LLM-mined relationships are generalizable world knowledge and are not contaminated by benchmark co-occurrence.
    Sec 3.4 and Appendix C.4 assume the CRG encodes diverse, generalizable relations; no contamination check is provided.
  • domain assumption Residual refinement of frozen CLIP patch features preserves open-vocabulary capabilities while improving localization.
    ARM adds residual Delta x to frozen CLIP features and relies on the L1 penalty to avoid catastrophic forgetting (Sec 3.1, 3.2).
  • domain assumption GATv2 adaptive attention can route useful relational information from CRG neighbors for both seen and unseen classes.
    ATM assumes adaptive aggregation over the CRG improves class representations; Sec 3.4.
  • domain assumption Evaluation splits and protocols from prior works are correctly applied.
    The paper reuses LESA/SDL/MKT splits and settings (Sec 4.1); errors in those protocols would propagate.
invented entities (1)
  • LLM-mined Class Relationship Graph (CRG)
    purpose: Provides the fixed graph structure for GAT-based inter-class transfer in the ATM.
    Constructed by querying gpt-4o-latest with benchmark class names; edges not released and contamination not assessed, yet central to the unseen-class gains.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DART: Dual Adaptive Refinement Transfer for Open-Vocabulary Multi-Label Recognition." pith.science (2026). https://pith.science/paper/LBWSWON5

@misc{pith2026250805585,
  author       = {Pith},
  title        = {Pith review of: DART: Dual Adaptive Refinement Transfer for Open-Vocabulary Multi-Label Recognition},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LBWSWON5}},
  note         = {Machine review of arXiv:2508.05585}
}
read the original abstract

Open-Vocabulary Multi-Label Recognition (OV-MLR) aims to identify multiple seen and unseen object categories within an image, requiring both precise intra-class localization to pinpoint objects and effective inter-class reasoning to model complex category dependencies. While Vision-Language Pre-training (VLP) models offer a strong open-vocabulary foundation, they often struggle with fine-grained localization under weak supervision and typically fail to explicitly leverage structured relational knowledge beyond basic semantics, limiting performance especially for unseen classes. To overcome these limitations, we propose the Dual Adaptive Refinement Transfer (DART) framework. DART enhances a frozen VLP backbone via two synergistic adaptive modules. For intra-class refinement, an Adaptive Refinement Module (ARM) refines patch features adaptively, coupled with a novel Weakly Supervised Patch Selecting (WPS) loss that enables discriminative localization using only image-level labels. Concurrently, for inter-class transfer, an Adaptive Transfer Module (ATM) leverages a Class Relationship Graph (CRG), constructed using structured knowledge mined from a Large Language Model (LLM), and employs graph attention network to adaptively transfer relational information between class representations. DART is the first framework, to our knowledge, to explicitly integrate external LLM-derived relational knowledge for adaptive inter-class transfer while simultaneously performing adaptive intra-class refinement under weak supervision for OV-MLR. Extensive experiments on challenging benchmarks demonstrate that our DART achieves new state-of-the-art performance, validating its effectiveness.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

78 extracted references · 58 canonical work pages

  1. [1]

    Hangbo Bao, Wenhui Wang, Li Dong, Qiang Liu, Owais Khan Mohammed, Kriti Aggarwal, Subhojit Som, Songhao Piao, and Furu Wei. 2022. Vlmo: Unified vision- language pre-training with mixture-of-modality-experts. NeurIPS 35 (2022), 32897–32912

    work page 2022

  2. [2]

    Avi Ben-Cohen, Nadav Zamir, Emanuel Ben-Baruch, Itamar Friedman, and Lihi Zelnik-Manor. 2021. Semantic diversity learning for zero-shot multi-label classi- fication. In ICCV. IEEE, 640–650

    work page 2021

  3. [3]

    Shaked Brody, Uri Alon, and Eran Yahav. 2022. How Attentive are Graph At- tention Networks?. In ICLR. OpenReview. https://openreview.net/forum?id= F72ximsx7C1

    work page 2022

  4. [4]

    Deli Chen, Yankai Lin, Wei Li, Peng Li, Jie Zhou, and Xu Sun. 2020. Measuring and Relieving the Over-Smoothing Problem for Graph Neural Networks from the Topological View. In AAAI. 3438–3445

    work page 2020

  5. [6]

    Long Chen, Wujing Zhan, Wei Tian, Yuhang He, and Qin Zou. 2019. Deep Integration: A Multi-Label Architecture for Road Scene Recognition. IEEE TIP 28, 10 (2019), 4883–4898. doi:10.1109/TIP.2019.2913079

    arXiv 2019

  6. [7]

    Tianshui Chen, Liang Lin, Riquan Chen, Xiaolu Hui, and Hefeng Wu. 2022. Knowledge-Guided Multi-Label Few-Shot Learning for General Image Recogni- tion. IEEE Trans. Pattern Anal. Mach. Intell. 44, 3 (2022), 1371–1384

    work page 2022

  7. [8]

    Tianshui Chen, Tao Pu, Hefeng Wu, Yuan Xie, Lingbo Liu, and Liang Lin. 2022. Cross-Domain Facial Expression Recognition: A Unified Evaluation Benchmark and Adversarial Graph Learning. IEEE Trans. Pattern Anal. Mach. Intell. 44, 12 (2022), 9887–9903

    work page 2022

  8. [9]

    Tianshui Chen, Muxin Xu, Xiaolu Hui, Hefeng Wu, and Liang Lin. 2019. Learning Semantic-Specific Graph Representation for Multi-Label Image Recognition. In ICCV. IEEE, 522–531. doi:10.1109/ICCV.2019.00061

    arXiv 2019

Show all 78 references
  1. [10]

    Zhao-Min Chen, Xiu-Shen Wei, Peng Wang, and Yanwen Guo. 2019. Multi-label image recognition with graph convolutional networks. In CVPR. 5177–5186

    2019

  2. [11]

    François Chollet. 2017. Xception: Deep learning with depthwise separable con- volutions. In Proceedings of the IEEE conference on computer vision and pattern recognition. 1251–1258

    2017

  3. [12]

    Tat-Seng Chua, Jinhui Tang, Richang Hong, Haojie Li, Zhiping Luo, and Yan- tao Zheng. 2009. NUS-WIDE: a real-world web image database from National University of Singapore. In Proceedings of the ACM International Conference on Image and Video Retrieval , Stéphane Marchand-Mail...

    2009

  4. [13]

    Runyu Ding, Jihan Yang, Chuhui Xue, Wenqing Zhang, Song Bai, and Xiaojuan Qi. 2023. PLA: Language-Driven Open-Vocabulary 3D Scene Understanding. In CVPR. IEEE, 7010–7019. doi:10.1109/CVPR52729.2023.00677

    2023

  5. [14]

    Yu Du, Fangyun Wei, Zihe Zhang, Miaojing Shi, Yue Gao, and Guoqi Li. 2022. Learning to prompt for open-vocabulary object detection with vision-language model. In CVPR. 14084–14093

    2022

  6. [15]

    James Foulds and Eibe Frank. 2010. A review of multi-instance learning assump- tions. The knowledge engineering review 25, 1 (2010), 1–25

    2010

  7. [16]

    Bin-Bin Gao and Hong-Yu Zhou. 2021. Learning to Discover Multi-Class Atten- tional Regions for Multi-Label Image Recognition. IEEE TIP 30 (2021), 5920–5932. doi:10.1109/TIP.2021.3088605

    2021

  8. [17]

    Golnaz Ghiasi, Xiuye Gu, Yin Cui, and Tsung-Yi Lin. 2022. Scaling open- vocabulary image segmentation with image-level labels. In ECCV. Springer, 540– 557

    2022

  9. [18]

    Yunchao Gong, Yangqing Jia, Thomas Leung, Alexander Toshev, and Sergey Ioffe

  10. [19]

    Sunan He, Taian Guo, Tao Dai, Ruizhi Qiao, Xiujun Shu, Bo Ren, and Shu-Tao Xia

  11. [20]

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-Rank Adaptation of Large Language Models. In International Conference on Learning Representations . https: //openreview.net/forum?id=nZeVKeeFYf9

    2022

  12. [21]

    Liang, Yupei Lin, Yandong Chen, Shanshan Zhong, Hefeng Wu, and Liang Lin

    Zhongzhan Huang, Guoming Ling, Vincent S. Liang, Yupei Lin, Yandong Chen, Shanshan Zhong, Hefeng Wu, and Liang Lin. 2025. RouterEval: A Comprehensive Benchmark for Routing LLMs to Explore Model-level Scaling Up in LLMs. ArXiv 2503.10657 (2025)

    2025 arXiv

  13. [22]

    Zhongzhan Huang, Guoming Ling, Shanshan Zhong, Hefeng Wu, and Liang Lin

  14. [23]

    Dat Huynh and Ehsan Elhamifar. 2020. A Shared Multi-Attention Framework for Multi-Label Zero-Shot Learning. In CVPR. 8773–8783

    2020

  15. [24]

    Dat Huynh, Jason Kuen, Zhe Lin, Jiuxiang Gu, and Ehsan Elhamifar. 2022. Open- vocabulary instance segmentation via robust cross-modal pseudo-labeling. In CVPR. 7020–7031

    2022

  16. [25]

    Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. 2021. Scaling up visual and vision-language representation learning with noisy text supervision. In ICML. PMLR, 4904–4916

    2021

  17. [26]

    Nagma Khan, Ushasi Chaudhuri, Biplab Banerjee, and Subhasis Chaudhuri. 2019. Graph convolutional network for multi-label VHR remote sensing scene recogni- tion. Neurocomputing 357 (2019), 36–46. doi:10.1016/J.NEUCOM.2019.05.024

    2019 doi

  18. [27]

    Alina Kuznetsova, Hassan Rom, Neil Alldrin, Jasper Uijlings, Ivan Krasin, Jordi Pont-Tuset, Shahab Kamali, Stefan Popov, Matteo Malloci, Alexander Kolesnikov, et al. 2020. The open images dataset v4: Unified image classification, object detection, and visual relationship detec...

    2020

  19. [28]

    Chung-Wei Lee, Wei Fang, Chih-Kuan Yeh, and Yu-Chiang Frank Wang. 2018. Multi-Label Zero-Shot Learning With Structured Knowledge Graphs. In CVPR. 1576–1585

    2018

  20. [29]

    Guofa Li, Zefeng Ji, Yunlong Chang, Shen Li, Xingda Qu, and Dongpu Cao. 2021. ML-ANet: A Transfer Learning Approach Using Adaptation Network for Multi- label Image Classification in Autonomous Driving. Chinese Journal of Mechanical Engineering 34 (12 2021). doi:10.1186/s10033-...

    2021 doi

  21. [30]

    Junnan Li, Ramprasaath Selvaraju, Akhilesh Gotmare, Shafiq Joty, Caiming Xiong, and Steven Chu Hong Hoi. 2021. Align before fuse: Vision and language repre- sentation learning with momentum distillation. NeurIPS 34 (2021), 9694–9705

    2021

  22. [31]

    Liunian Harold Li, Mark Yatskar, Da Yin, Cho-Jui Hsieh, and Kai-Wei Chang. 2019. VisualBERT: A Simple and Performant Baseline for Vision and Language. arXiv preprint abs/1908.03557 (2019). arXiv:1908.03557 http://arxiv.org/abs/1908.03557

    2019 arXiv

  23. [32]

    Xirong Li, Shuai Liao, Weiyu Lan, Xiaoyong Du, and Gang Yang. 2015. Zero- shot Image Tagging by Hierarchical Semantic Embedding. In Proceedings of the International ACM SIGIR Conference on Research and Development in Information Retrieval, Ricardo Baeza-Yates, Mounia Lalmas, A...

    2015

  24. [33]

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. 2014. Microsoft coco: Common objects in context. In Computer vision–ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proce...

    2014

  25. [34]

    Yuqi Lin, Minghao Chen, Kaipeng Zhang, Hengjia Li, Mingming Li, Zheng Yang, Dongqin Lv, Binbin Lin, Haifeng Liu, and Deng Cai. 2024. Tagclip: A local-to- global framework to enhance open-vocabulary multi-label classification of clip without training. In Proceedings of the AAAI...

    2024

  26. [35]

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2023. Improved Baselines with Visual Instruction Tuning. ArXiv 2310.03744 (2023)

    2023 arXiv

  27. [36]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. [n. d.]. Visual In- struction Tuning. In Annual Conference on Neural Information Processing Systems, NeurIPS, New Orleans, LA, USA, December 10-16, 2023

    2023

  28. [37]

    Haijing Liu, Tao Pu, Hefeng Wu, Keze Wang, and Liang Lin. 2024. Category- Adaptive Cross-Modal Semantic Refinement and Transfer for Open-Vocabulary Multi-Label Recognition. ArXiv 2412.06190 (2024)

    2024 arXiv

  29. [38]

    Ziming Liu, Song Guo, Xiaocheng Lu, Jingcai Guo, Jiewei Zhang, Yue Zeng, and Fushuo Huo. 2023. (ML)2P-Encoder: On Exploration of Channel-Class Correlation for Multi-Label Zero-Shot Learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition ....

    2023

  30. [39]

    Ilya Loshchilov and Frank Hutter. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 (2017)

    2017 arXiv

  31. [40]

    Peirong Ma, Zhiquan He, Wu Ran, and Hong Lu. 2023. A transferable generative framework for multi-label zero-shot learning. IEEE Transactions on Circuits and Systems for Video Technology 34, 5 (2023), 3409–3423

    2023

  32. [41]

    Namrutha Sridhar, K

    B.V. Namrutha Sridhar, K. Mrinalini, and P. Vijayalakshmi. 2020. Data An- notation and Multi-Emotion Classification for Social Media Text. In Interna- tional Conference on Communication and Signal Processing (ICCSP) . 1011–1015. doi:10.1109/ICCSP48568.2020.9182362

    2020

  33. [42]

    Khan, Fahad Shahbaz Khan, Ling Shao, and Mubarak Shah

    Sanath Narayan, Akshita Gupta, Salman H. Khan, Fahad Shahbaz Khan, Ling Shao, and Mubarak Shah. 2021. Discriminative Region-based Multi-Label Zero-Shot Learning. In ICCV. 8711–8720

    2021

  34. [43]

    OpenAI. 2023. GPT-4 technical report. ArXiv 2303.08774 (2023)

    2023 arXiv

  35. [44]

    Funkhouser

    Songyou Peng, Kyle Genova, Chiyu Max Jiang, Andrea Tagliasacchi, Marc Polle- feys, and Thomas A. Funkhouser. 2023. OpenScene: 3D Scene Understanding 9 MM ’25, October 27–31, 2025, Dublin, Ireland Haijing Liu, Tao Pu, Hefeng Wu, Keze Wang, Liang Lin with Open Vocabularies. In I...

    2023

  36. [45]

    Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. Glove: Global Vectors for Word Representation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, EMNLP 2014 , Alessandro Moschitti, Bo Pang, and Walter Daelemans (Eds.). 1532–1543

    2014

  37. [46]

    Tao Pu, Tianshui Chen, Hefeng Wu, Yongyi Lu, and Liang Lin. 2024. Spatial- Temporal Knowledge-Embedded Transformer for Video Scene Graph Generation. IEEE Trans. Image Process. 33 (2024), 556–568

    2024

  38. [47]

    Tao Pu, Qianru Lao, Hefeng Wu, Tianshui Chen, Ling Tian, Jie Liu, and Liang Lin. 2024. Category-Adaptive Label Discovery and Noise Rejection for Multi- Label Recognition With Partial Positive Labels. IEEE Trans. Multim. 26 (2024), 9591–9602

    2024

  39. [48]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. In ICML, Vol. ...

    2021

  40. [49]

    Shafin Rahman and Salman Khan. 2018. Deep multiple instance learning for zero- shot image tagging. In Asian Conference on Computer Vision . Springer, 530–546

    2018

  41. [50]

    Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedan- tam, Devi Parikh, and Dhruv Batra

    Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedan- tam, Devi Parikh, and Dhruv Batra. 2017. Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization. In ICCV. IEEE, 618–626. doi:10.1109/ICCV.2017.74

    2017 doi

  42. [51]

    Jie Tao and Xing Fang. 2020. Toward multi-label sentiment analysis: a transfer learning based approach. J. Big Data 7, 1 (2020), 1. doi:10.1186/S40537-019-0278-0

    2020 doi

  43. [52]

    Qwen Team. 2024. Qwen2 Technical Report. ArXiv 2407.10671 (2024)

    2024 arXiv

  44. [53]

    Qwen Team. 2024. Qwen2.5: A Party of Foundation Models. https://qwenlm. github.io/blog/qwen2.5/

    2024

  45. [54]

    LLaMA 3 Teams. 2024. The Llama 3 Herd of Models. arXiv:2407.21783 [cs.AI] https://arxiv.org/abs/2407.21783

    2024 arXiv

  46. [55]

    Belongie

    Andreas Veit, Neil Alldrin, Gal Chechik, Ivan Krasin, Abhinav Gupta, and Serge J. Belongie. 2017. Learning from Noisy Large-Scale Datasets with Minimal Super- vision. In CVPR. IEEE, 6575–6583. doi:10.1109/CVPR.2017.696

    2017 doi

  47. [56]

    Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph Attention Networks. In ICLR. OpenReview. https://openreview.net/forum?id=rJXMpikCZ

    2018

  48. [57]

    Zhouxia Wang, Tianshui Chen, Guanbin Li, Ruijia Xu, and Liang Lin. 2017. Multi- label image recognition by recurrently discovering attentional regions. In ICCV. 464–472

    2017

  49. [58]

    Yunchao Wei, Wei Xia, Min Lin, Junshi Huang, Bingbing Ni, Jian Dong, Yao Zhao, and Shuicheng Yan. 2015. HCP: A flexible CNN framework for multi-label image classification. IEEE TPAMI 38, 9 (2015), 1901–1907

    2015

  50. [59]

    Hefeng Wu, Weifeng Chen, Zhibin Liu, Tianshui Chen, Zhiguang Chen, and Liang Lin. 2024. Contrastive Transformer Learning With Proximity Data Generation for Text-Based Person Search.IEEE Trans. Circuits Syst. Video Technol. 34, 8 (2024), 7005–7016

    2024

  51. [60]

    Hefeng Wu, Yandong Chen, Lingbo Liu, Tianshui Chen, Keze Wang, and Liang Lin

  52. [61]

    Hefeng Wu, Guangzhi Ye, Ziyang Zhou, Ling Tian, Qing Wang, and Liang Lin

  53. [62]

    Jin Yuan, Shikai Chen, Yao Zhang, Zhongchao Shi, Xin Geng, Jianping Fan, and Yong Rui. 2023. Graph Attention Transformer Network for Multi-label Image Classification. ACM Trans. Multim. Comput. Commun. Appl. 19, 4 (2023), 150:1– 150:16. doi:10.1145/3578518

    2023 doi

  54. [63]

    Yuhang Zang, Wei Li, Kaiyang Zhou, Chen Huang, and Chen Change Loy. 2022. Open-vocabulary detr with conditional matching. In ECCV. Springer, 106–122

    2022

  55. [64]

    Alireza Zareian, Kevin Dela Rosa, Derek Hao Hu, and Shih-Fu Chang. 2021. Open-vocabulary object detection using captions. In CVPR. 14393–14402

    2021

  56. [65]

    IEEE Trans

    SQLNet: Scale-Modulated Query and Localization Network for Few-Shot Class-Agnostic Counting. IEEE Trans. Image Process. 34 (2025), 4631–4645

    2025

  57. [66]

    Yang Zhang, Boqing Gong, and Mubarak Shah. 2016. Fast zero-shot image tagging. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . 5985–5994

    2016

  58. [67]

    Xuelin Zhu, Jian Liu, Weijia Liu, Jiawei Ge, Bo Liu, and Jiuxin Cao. 2023. Scene- Aware Label Graph Learning for Multi-Label Image Classification. InICCV. 1473– 1482. 10 Supplementary Material MM ’25, October 27–31, 2025, Dublin, Ireland Supplementary Material for DART: Dual A...

    2023

  59. [71]

    Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. 2023. Sig- moid loss for language image pre-training. In Proceedings of the IEEE/CVF inter- national conference on computer vision . 11975–11986

    2023

  60. [74]

    computer

    Consequently, both the model-predicted responsibility ˆ𝑧𝑖,𝑐 and the prior responsibility ˆ𝑧∗ 𝑖,𝑐 are 0 for negative classes. The final smoothed responsibility is also zero: ˆ𝑧′ 𝑖,𝑐 = 0 if𝑦𝑐 = 0. (C.1) Case 2: Positive Bag (𝑦𝑐 = 1) We need𝑃(𝑧𝑖,𝑐 = 1|𝑦𝑐 = 1,𝑏 ;𝜃). Applying Bayes...

    2025

  61. [75]

    Synonymy/Similarity: Two categories are concep- tually very similar or synonymous

  62. [76]

    Is-a/Hypernym: One category is a superordinate or subordinate concept of the other

  63. [77]

    Functional Relationship: The function or use of one category is related to the other

  64. [78]

    Co-occurrence: Two categories often appear in the same context or environment

  65. [79]

    Natural” is conceptually very similar to “nature

    Part-Whole Relationship: One category is a com- ponent of the other. Instructions. Please provide the information for each rele- vant category in the following format: Related Category [Number]: [Category Name] • Type of Relationship: [Relationship Type] • Association Strength...

    2025

  66. [816]

    doi:10.1609/AAAI.V37I1.25159

    doi

  67. [2014]

    In ICLR, Yoshua Bengio and Yann LeCun (Eds.)

    Deep Convolutional Ranking for Multilabel Image Annotation. In ICLR, Yoshua Bengio and Yann LeCun (Eds.). http://arxiv.org/abs/1312.4894

    arXiv

  68. [2023]

    In AAAI, Brian Williams, Yiling Chen, and Jennifer Neville (Eds.)

    Open-Vocabulary Multi-Label Classification via Multi-Modal Knowledge Transfer. In AAAI, Brian Williams, Yiling Chen, and Jennifer Neville (Eds.). 808–

  69. [2024]

    IEEE Trans

    Dual-View Data Hallucination With Semantic Relation Guidance for Few- Shot Image Recognition. IEEE Trans. Multim. 26 (2024), 11302–11315

    2024

  70. [2025]

    MiniLongBench: The Low-cost Long Context Understanding Benchmark for Large Language Models. In ACL. 11442–11460

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.