Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

Negation-Aware Test-Time Adaptation for Vision-Language Models

T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A test-time tweak to normalization layers gives vision-language models a grasp of negation.

desk verdict The empirical story is solid, but the derivation of L_sr in Eq. (7) is internally inconsistent and needs to be fixed before publication. read the letter →

arxiv 2507.19064 v2 pith:4CNNQRV4 submitted 2025-07-25 cs.CV

Haochen Han , Alex Jinpeng Wang , Fangming Liu , Jun Zhu This is my paper · ORCID
classification cs.CV
keywords negationunderstandingvision-languagemodelstest-timeadaptationLayerNormdual-conceptshiftcontrastivelearningcross-modalretrievalCLIP
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that vision-language models (VLMs) fail at negation not mainly because negation is rare in their training data, but because the embedding distributions of affirmative, negation-conditioned, and reversed negation-conditioned text are misaligned: the model puts semantically consistent affirmative and negated captions far apart, while placing semantically opposite captions close together. It proposes NEAT, a test-time adaptation method that updates only the LayerNorm parameters of the text encoder to pull these distributions into line. With less than 0.01% of trainable parameters and only unlabeled test data, NEAT matches or beats post-training methods that fine-tune on tens of millions of negation-labelled pairs, and the adapted layers transfer to unseen datasets and to other vision-language models. A sympathetic reader would care because the method points to a cheap, data-free route to a capability that otherwise requires massive curated data collection.

What carries the argument

The load-bearing object is the dual-concept shift diagnosis, made quantitative with Maximum Mean Discrepancy on CIFAR10 features: normal versus negation-conditioned text embeddings show MMD = 1.57, while negation-conditioned versus reversed text embeddings show MMD = 0.08. The adaptation machinery is NEAT's combined objective, a sum of three losses over the text encoder's LayerNorm parameters: refined entropy minimization on a negative-penalised retrieval score, a reversion loss that reduces (for each reversed caption) to a simple metric gap between similarity to its paired image and similarity to the single hardest negative visual sample, and a triplet loss on textual embeddings. The only non-distributional ingredient is an LLM that parses each negated caption into affirmative sub-captions and generates the reversed caption; the actual learned parameters are just the scale and shift of normalization layers.

What would settle it

Run NEAT on a negation benchmark while replacing the hardest-negative selection in the reversion loss with a randomly chosen sample from the same batch; if retrieval and multiple-choice performance do not drop, the hardest-negative mechanism is not doing the claimed work, and the geometric diagnosis would need revision.

Watch

Extended reading notes

Core claim

Negation understanding in a VLM can be repaired at inference time by correcting two distributional errors measured in the text encoder's embedding space: the affinity gap between an affirmative caption and its negation-conditioned variant (which share meaning), and the false affinity between a negation-conditioned caption and its reversed counterpart (which are opposites). NEAT decomposes each negated caption into a positive and a negative part with an LLM, uses the negative part to re-rank candidate visual matches, then optimizes three objectives on the LayerNorm parameters of the text encoder: an entropy loss over refined text–image matches, a semantics-reversion loss that treats the reversed caption as a hard negative against its paired image, and a textual loss that pulls negation-conditioned text toward its affirmative base while pushing it away from its reversed form. The paper shows empirically that this adaptation improves retrieval and multiple-choice accuracy on negation benchmarks across images, videos, and chest X-rays, and that the modified normalization layers transfer to new negation tasks and to different base models.

Load-bearing premise

The method assumes that the single 'hardest' visual sample for a reversed caption, meaning the image whose embedding is least similar to that caption, is a semantically sound contrastive target, so that pushing the reversed caption away from it teaches the model to separate opposites; if that sample is actually an unrelated image, the loss pulls the representation toward noise.

Editorial extensions

If this is right

  • Negation capability can be added to an existing VLM without collecting or labelling negation data; adaptation runs on the unlabeled query set at deployment time.
  • Model updates are tiny (normalization parameters only), so adaptation is cheap enough to run per client or per domain, and the adapted layers can be shipped as a small patch or swapped between models.
  • Because the method targets distribution geometry rather than memorised negation patterns, its gains transfer to negation queries and benchmarks the model never saw during adaptation.
  • Post-training pipelines that generate tens of millions of negated captions may be unnecessary for many applications; a test-time correction can reach comparable accuracy with a fraction of the data and parameters.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same distribution-shift repair could plausibly apply to other closed-class linguistic operators, such as quantifiers, temporal markers, or modality words, where VLMs behave like bag-of-words models; the paper only demonstrates negation, but the mechanism of aligning consistent paraphrases and separating semantic opposites is operator-agnostic.
  • The method's reliance on an LLM to separate positive and negative components and build reversed captions means its ceiling is partly set by the LLM's parse quality; a testable extension would measure NEAT's sensitivity to parse errors or replace the LLM with a deterministic parser.
  • The reversion loss selects a single hardest negative visual sample; a natural variant would be to sample a small set of hardest negatives, which might retain the anti-clustering benefit while reducing sensitivity to one mis-selected anchor.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes NEAT, a test-time adaptation method that updates only LayerNorm parameters of the text encoder of a vision-language model in order to improve negation understanding. The method uses an LLM to split each negation-conditioned caption into positive and negative components, selects visual candidates through a penalty-augmented similarity, and optimizes three objectives: refined entropy (L_ent), a semantics-reversion contrastive loss (L_sr), and a textual dual-concept debiasing loss (L_tri). Experiments on COCO, VOC2007, MSR-VTT, CheXpert, and nine image classification datasets report consistent gains over CLIP, NegCLIP, and BLIP baselines, and competitive or superior performance relative to post-training methods that use millions of image-text pairs, while updating only a tiny fraction of parameters. The paper also reports transfer of the adapted normalization layers to unseen datasets and to a medical foundation model.

Significance. If the empirical results are taken at face value, the paper identifies a practical and relatively underexplored problem, proposes a lightweight adaptation procedure, and validates it across image, video, and medical benchmarks with released code. The finding that updating only LayerNorm parameters of the text encoder can produce large negation-understanding gains and transfer across datasets would be a useful contribution to the test-time adaptation and vision-language communities. However, the theoretical justification of the central loss in Eq. (7) is not sound as written, the negative-sample selection is under-specified, and one of the headline transfer metrics (reversed negation-conditioned error, Table VI) is directly optimized by the method; these issues currently prevent me from fully endorsing the paper's claims.

major comments (4)
  1. [IV-B2, Fig. 4] The derivation of the simplified metric form in Eq. (7) uses the approximation -log(1+exp(x)) ≈ -x, which is accurate only for x large and positive. With \hat v_i^- chosen as the visual sample of minimal similarity to \bar t_i, and with \hat v_i included in the comparison set, we have x = (\bar t_i · \hat v_i^- - \bar t_i · \hat v_i)/τ2 ≤ 0, so the approximation is invalid exactly in the intended operating regime. In this regime the exact loss has derivative -sigmoid(x) with respect to x, with magnitude at most 1/2, whereas the linearized objective has constant gradient, so the two objectives are not equivalent. Because Table IX shows that L_sr is the component that raises MCQ accuracy from 28.69 to 45.91, the paper should state clearly which form is implemented (the exact log-sum-exp loss or the linearized metric loss) and justify the choice; if the linearized form is used, the claimed relationship to the negative InfoNCE in Eq. (6) does not hold.
  2. [IV-B2, Fig. 4] The selection rule for \hat v_i^- is under-specified and semantically questionable. The paper calls \hat v_i^- the 'hardest' visual sample but then defines it as the one with minimal similarity to \bar t_i, which is the opposite of the standard hardest-negative convention; it is also not stated whether \hat v_i itself is included in the candidate set, how large the candidate set is, or how ties are broken. More substantively, for a reversed caption such as 'a person in sight, no dog in the grass', the least-similar CLIP embedding is unlikely to be the semantically correct contrastive target, and minimizing Eq. (7) can pull \bar t_i toward an arbitrary unrelated image while pushing it away from the paired image. I request a precise definition of the candidate set, statistics on the selected samples (for example, their semantic categories and similarity ranks), and an ablation that replaces the minimal-similarity sample with a standard hardest negative drawn from the same batch.
  3. [V-C, Table VI] The reversed negation-conditioned error reported in Table VI is not an independent measure of generalization. The second term of L_tri and the L_sr loss are explicitly designed to separate the reversed negation-conditioned caption from the negation-conditioned caption and from the paired image, so the near-perfect RNC error of 1.36% after adaptation on MS-COCO is partly a self-fulfilling optimization target rather than evidence of transfer. The generalization claims should be supported primarily by metrics that are not directly optimized, such as Retrieval-Neg on MSR-VTT, MCQ-Neg on VOC2007, and affirmative classification tasks, and the RNC numbers should be accompanied by evaluation on held-out negation distributions with non-template structure.
  4. [IV-B3, Eq. (8)] In Eq. (8), the term (2 - \|\hat t_i - \bar t_i\|_2) is described as encouraging maximum distance between \hat t_i and \bar t_i, but the expression is unbounded below as the distance grows, and the scale of this term relative to the first term depends on the units of the embedding norm. If a squared distance was intended, the notation should be \|\cdot\|_2^2, and the boundedness and interaction with λ should be clarified; otherwise the optimization may be dominated by a small number of large-distance examples.
minor comments (6)
  1. [I and Abstract] The abstract states 'less than 0.01% of trainable parameters', while the introduction states '0.014‰' of trainable parameters and '0.36‰' of the data; these numbers should be reconciled.
  2. [V-B1] The text says 'From Fig. V' when referring to the results shown in Table V; the reference should be corrected.
  3. [Table VI] The dataset name 'Caltch101' should be spelled 'Caltech101'.
  4. [II-A and Table II] The method name is spelled inconsistently as 'ConCLIP' in Table II and 'CoNCLIP' in Section II-A and the references; one spelling should be used throughout.
  5. [IV-B1, Eq. (4)] The quantity S(\hat t_i, v_j) is called a similarity but is not guaranteed to be non-negative; it would be clearer to describe it as a scoring function rather than a similarity.
  6. [IV-B1, Eqs. (2) and (5)] Equation (2) sums over N_T while Eq. (5) sums over the batch size B; the batch-level normalization and the role of N_T should be stated explicitly.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity found; NEAT's losses are stated objectives evaluated on external held-out benchmarks.

full rationale

NEAT's three losses are defined directly from the stated negation-understanding objectives in Section IV-B: entropy refinement over negation-separated captions (Eq. 5), reversed-caption contrastive separation (Eq. 7), and textual dual-concept debiasing (Eq. 8). None of these losses is fitted to the reported benchmark metrics; the parameters are adapted on unlabeled MS-COCO Retrieval-Neg data and then evaluated on held-out datasets (VOC2007, CIFAR10/100, ImageNet variants, CheXpert, etc.). The reversed negation-conditioned error rate in Table VI is aligned with what L_sr and the second term of L_tri are designed to optimize, but because the evaluation is on unseen datasets after adaptation on a different corpus, the result is a genuine transfer measurement rather than a fitted input being renamed as a prediction. There are no load-bearing self-citations: the most relevant methodological citation ([25], TCR) is to an external group, and the only author self-citation ([39]) is a non-essential related-work reference. One mathematical concern lies outside the circularity scope: the approximation in Eq. (7) replacing -log(1+e^x) by -x is not valid when the selected hardest negative has lower similarity than the paired positive (so the exponent is non-positive), making the claimed equivalence to a metric loss suspect; this is a correctness issue, not a circular reduction. Overall, the central claims are self-contained and externally benchmarked, so no circularity is present.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

No new physical or conceptual entities are introduced. The method relies on the pre-trained VLM, an external LLM, and standard losses; the only 'invention' is the combination of existing components for negation-specific TTA.

free parameters (4)
  • alpha = 1.0
    Trade-off parameter in Eq (4) controlling the penalty for negative components in the refined similarity score S(hat t_i, v_j). Set to 1 for all experiments.
  • lambda = 5.0
    Balance hyperparameter in Eq (8) weighting the textual debiasing term. Chosen by validation; sensitivity analysis in Fig 5b shows stability for 5-20.
  • tau1 = 0.03 (image) / 0.07 (video)
    Temperature for the refined entropy loss in Eq (5). Hand-set per domain.
  • tau2 = 0.07 (image) / 1.0 (video)
    Temperature for the semantics reversion loss in Eq (7). Hand-set per domain; sensitivity shown in Fig 5c.
assumptions (4)
  • domain assumption Llama-3-8B accurately decomposes negated captions into positive and negative affirmative components
    Section IV-B1 relies on an LLM to split captions such as 'a photo of a dog not on grass' into 'a photo of a dog' and 'a photo of grass'. No error analysis or prompt template is provided.
  • domain assumption Negation understanding can be modeled entirely as a text-side distribution shift
    Section III-B defines the problem as P(T_test) != P(T_train), and NEAT adjusts only text-encoder LayerNorm parameters, leaving the visual encoder fixed. This assumes the shift is localized in text embedding statistics.
  • ad hoc to paper The visual sample with minimal similarity to the reversed caption is a valid semantic contrast target
    Section IV-B2 selects 'the hardest visual sample hat v_i^- that shares minimal similarity with the reversed text' and uses it as the negative in L_sr. This selection rule is not derived or validated independently.
  • domain assumption LayerNorm adaptation is sufficient to close the dual-concept gap
    The method updates only normalization layers of the text encoder (Section IV-B). The paper provides empirical support but no theoretical justification for why this captures the negation shift.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Negation-Aware Test-Time Adaptation for Vision-Language Models." pith.science (2026). https://pith.science/paper/4CNNQRV4

@misc{pith2026250719064,
  author       = {Pith},
  title        = {Pith review of: Negation-Aware Test-Time Adaptation for Vision-Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4CNNQRV4}},
  note         = {Machine review of arXiv:2507.19064}
}
read the original abstract

In this paper, we study a practical but less-touched problem in Vision-Language Models (VLMs), \ie, negation understanding. Specifically, many real-world applications require models to explicitly identify what is false or non-existent, \eg, radiologists may search for images that exclude specific conditions. Despite the impressive transferability of VLMs through large-scale training, they suffer from a critical limitation that fails to handle negation. To address this challenge, existing methods attribute its root cause to the scarcity of negation training data and propose to fine-tune VLMs on massive data containing explicit negation. Undoubtedly, such data-centric solutions demand substantial data and computational resources, limiting their sustainable widespread adoption. To tackle negation in a low-carbon manner, we empirically observe that the key obstacle lies in the dual-concept shifts between the affirmation and negation distributions. Therefore, we propose a Negation-Aware Test-Time Adaptation (NEAT) method to efficiently adjust distribution-related parameters during inference. In brief, NEAT can reduce distribution shift in consistent semantics while eliminating false distributional consistency in unrelated semantics. Extensive experiments on the various negation understanding tasks verify the effectiveness of the proposed method. Remarkably, with less than 0.01\% of trainable parameters, NEAT achieves comparable or superior performance to state-of-the-art post-training approaches. Our code is available at https://github.com/hhc1997/NEAT.

Figures

Figures reproduced from arXiv: 2507.19064 by the authors.

Figure 1
Figure 1. A toy example to illustrate the dual-concept shifts problem in Vision￾Language Models when understanding negation. Negation-conditioned (NC) texts negate absent elements and share consistent semantics with affirmative ones. Reversed negation-conditioned (RNC) texts wrongly affirm absent elements and negate present ones, which are semantically opposite to NC texts. Although CLIP learns diverse open-world knowledge, i… view at source ↗
Figure 2
Figure 2. Dual-concept shifts problem observed in pre-trained VLMs when understanding negation. Distributions of normal text, negation-conditioned text, and reversed negation-conditioned text of the test set of CIFAR10 are shown on the left, middle, and right, respectively. All embeddings are produced by the OpenAI CLIP and visualized with UMAP [30] for dimension reduction. The Maximum Mean Discrepancy (MMD) metrics are also … view at source ↗
Figure 3
Figure 3. Overview of the proposed NEAT. Given the test data with negation contexts, a LLM is first employed to separate the negation forms and generate [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: A toy example to show the challenge of semantic-reversed contrastive [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 7
Figure 7. Figure 7: In brief, the original validation set of MS-COCO [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]
Figure 5
Figure 5. Figure 5: Finer-grained ablation studies. (a) The top-1 prediction accuracy on the vanilla KNN-based prediction and our negation-separated prediction. (b) The [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: Comparison of discrimination capability across different negation types. This figure shows the similarity distributions of positive and negative pairs [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]
Figure 7
Figure 7. Figure 7: Case studies of our NEAT in understanding negation. (a) Multiple [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Adapting Vision-Language Models Without Labels: A Comprehensive Survey

    cs.LG 2025-08 conditional novelty 5.0 of 10

    A survey that organizes unsupervised vision-language model adaptation by unlabeled-data availability into four paradigms: data-free transfer, domain transfer, episodic test-time, and online test-time adaptation.

Reference graph

Works this paper leans on

51 extracted references · 30 canonical work pages · cited by 1 Pith paper

  1. [1]

    Leveraging vision-language models for improving domain generalization in image classification

    Sravanti Addepalli, Ashish Ramayee Asokan, Lakshay Sharma, and R Venkatesh Babu. Leveraging vision-language models for improving domain generalization in image classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 23922–23932, 2024

    work page 2024

  2. [2]

    Vision-language models do not understand negation

    Kumail Alhamoud, Shaden Alshammari, Yonglong Tian, Guohao Li, Philip Torr, Yoon Kim, and Marzyeh Ghassemi. Vision-language models do not understand negation. arXiv preprint arXiv:2501.09425 , 2025

    arXiv 2025

  3. [3]

    Foundation models defining a new era in vision: a survey and outlook

    Muhammad Awais, Muzammal Naseer, Salman Khan, Rao Muhammad Anwer, Hisham Cholakkal, Mubarak Shah, Ming-Hsuan Yang, and Fahad Shahbaz Khan. Foundation models defining a new era in vision: a survey and outlook. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025

    2025

  4. [4]

    Conceptual 12m: Pushing web-scale image-text pre-training to recognize long-tail visual concepts

    Soravit Changpinyo, Piyush Sharma, Nan Ding, and Radu Soricut. Conceptual 12m: Pushing web-scale image-text pre-training to recognize long-tail visual concepts. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 3558–3568, 2021

    work page 2021

  5. [5]

    Remote sensing image scene classification: Benchmark and state of the art

    Gong Cheng, Junwei Han, and Xiaoqiang Lu. Remote sensing image scene classification: Benchmark and state of the art. Proceedings of the IEEE, 105(10):1865–1883, 2017

    2017

  6. [6]

    An analysis of single-layer networks in unsupervised feature learning

    Adam Coates, Andrew Ng, and Honglak Lee. An analysis of single-layer networks in unsupervised feature learning. In Proceedings of the four- teenth international conference on artificial intelligence and statistics , pages 215–223. JMLR Workshop and Conference Proceedings, 2011

    work page 2011

  7. [7]

    look! it is not a bamoule!

    Alex de Carvalho, C ´ecile Crimon, Axel Barrault, John Trueswell, and Anne Christophe. “look! it is not a bamoule!”: 18-and 24-month-olds can use negative sentences to constrain their interpretation of novel word meanings. Developmental science, 24(4):e13085, 2021

    work page 2021

  8. [8]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei- Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition , pages 248–255. Ieee, 2009

    2009

Show all 51 references
  1. [9]

    The llama 3 herd of models

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv e-prints , pages arXiv–2407, 2024

    2024

  2. [10]

    The pascal visual object classes (voc) challenge

    Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. The pascal visual object classes (voc) challenge. International journal of computer vision , 88:303–338, 2010

    2010

  3. [11]

    One-shot learning of object categories

    Li Fei-Fei, Robert Fergus, and Pietro Perona. One-shot learning of object categories. IEEE transactions on pattern analysis and machine intelligence, 28(4):594–611, 2006

    2006

  4. [12]

    Diverse data augmentation with diffusions for effective test-time prompt tuning

    Chun-Mei Feng, Kai Yu, Yong Liu, Salman Khan, and Wangmeng Zuo. Diverse data augmentation with diffusions for effective test-time prompt tuning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2704–2714, 2023

    2023

  5. [13]

    Tuning vision-language models with multiple prototypes clustering

    Meng-Hao Guo, Yi Zhang, Tai-Jiang Mu, Sharon X Huang, and Shi-Min Hu. Tuning vision-language models with multiple prototypes clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence , 2024

    2024

  6. [14]

    Dual alignment unsupervised domain adaptation for video-text retrieval

    Xiaoshuai Hao, Wanqian Zhang, Dayan Wu, Fei Zhu, and Bo Li. Dual alignment unsupervised domain adaptation for video-text retrieval. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 18962–18972, 2023

    2023

  7. [15]

    Does understanding negation entail affirmation?: An examination of negated metaphors

    Uri Hasson and Sam Glucksberg. Does understanding negation entail affirmation?: An examination of negated metaphors. Journal of Prag- matics, 38(7):1015–1032, 2006

    2006

  8. [16]

    Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification

    Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , 12(7):2217–2226, 2019

    2019

  9. [17]

    A visual–language foundation model for pathology image analysis using medical twitter

    Zhi Huang, Federico Bianchi, Mert Yuksekgonul, Thomas J Montine, and James Zou. A visual–language foundation model for pathology image analysis using medical twitter. Nature medicine , 29(9):2307– 2316, 2023

    2023

  10. [18]

    Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison

    Jeremy Irvin, Pranav Rajpurkar, Michael Ko, Yifan Yu, Silviana Ciurea- Ilcus, Chris Chute, Henrik Marklund, Behzad Haghgoo, Robyn Ball, Katie Shpanskaya, et al. Chexpert: A large chest radiograph dataset with uncertainty labels and expert comparison. In Proceedings of the AAAI...

    2019

  11. [19]

    Scaling up visual and vision-language representation learning with noisy text supervision

    Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representation learning with noisy text supervision. In International conference on machine learning , pages 4904–4916...

    2021

  12. [20]

    Efficient test-time adaptation of vision-language models

    Adilbek Karmanov, Dayan Guan, Shijian Lu, Abdulmotaleb El Saddik, and Eric Xing. Efficient test-time adaptation of vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14162–14171, 2024

    2024

  13. [21]

    Wilds: A benchmark of in-the- wild distribution shifts

    Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, et al. Wilds: A benchmark of in-the- wild distribution shifts. In International conference on machine learning, ...

    2021

  14. [22]

    Mitigating test-time bias for fair image retrieval

    Fanjie Kong, Shuai Yuan, Weituo Hao, and Ricardo Henao. Mitigating test-time bias for fair image retrieval. Advances in Neural Information Processing Systems, 36, 2024

    2024

  15. [23]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009

    2009

  16. [24]

    Ra-tta: Retrieval-augmented test-time adaptation for vision-language models

    Youngjun Lee, Doyoung Kim, Junhyeok Kang, Jihwan Bang, Hwanjun Song, and Jae-Gil Lee. Ra-tta: Retrieval-augmented test-time adaptation for vision-language models. In The Thirteenth International Conference on Learning Representations , 2025

    2025

  17. [25]

    Test-time adaptation for cross-modal retrieval with query shift

    Haobin Li, Peng Hu, Qianjun Zhang, Xi Peng, Xiting Liu, and Mouxing Yang. Test-time adaptation for cross-modal retrieval with query shift. arXiv preprint arXiv:2410.15624 , 2024

    2024 arXiv

  18. [26]

    Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In ICML, 2022

    2022

  19. [27]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer vision–ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceeding...

    2014

  20. [28]

    Moil: Momentum imitation learning for efficient vision-language adaptation

    Gen Luo, Yiyi Zhou, Minglang Huang, Tianhe Ren, Xiaoshuai Sun, and Rongrong Ji. Moil: Momentum imitation learning for efficient vision-language adaptation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

    2024

  21. [29]

    Crepe: Can vision-language foundation models reason compositionally? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10910–10921, 2023

    Zixian Ma, Jerry Hong, Mustafa Omer Gul, Mona Gandhi, Irena Gao, and Ranjay Krishna. Crepe: Can vision-language foundation models reason compositionally? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10910–10921, 2023

    2023

  22. [30]

    Umap: Uniform manifold approximation and projection for dimension reduction

    Leland McInnes, John Healy, and James Melville. Umap: Uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv:1802.03426, 2018

    2018 arXiv

  23. [31]

    Towards stable test-time adaptation in dynamic wild world

    Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Zhiquan Wen, Yaofo Chen, Peilin Zhao, and Mingkui Tan. Towards stable test-time adaptation in dynamic wild world. In The Eleventh International Conference on Learning Representations, 2023

    2023

  24. [32]

    Representation learn- ing with contrastive predictive coding

    Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learn- ing with contrastive predictive coding. arXiv preprint arXiv:1807.03748, 2018

    2018 arXiv

  25. [33]

    Know” no”better: A data-driven approach for enhancing negation awareness in clip

    Junsung Park, Jungbeom Lee, Jongyoon Song, Sangwon Yu, Dahuin Jung, and Sungroh Yoon. Know” no”better: A data-driven approach for enhancing negation awareness in clip. arXiv preprint arXiv:2501.10913, 2025

    2025

  26. [34]

    Cats and dogs

    Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and CV Jawahar. Cats and dogs. In 2012 IEEE conference on computer vision and pattern recognition, pages 3498–3505. IEEE, 2012

    2012

  27. [35]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning , p...

    2021

  28. [36]

    Test-time prompt tuning for zero- shot generalization in vision-language models

    Manli Shu, Weili Nie, De-An Huang, Zhiding Yu, Tom Goldstein, Anima Anandkumar, and Chaowei Xiao. Test-time prompt tuning for zero- shot generalization in vision-language models. Advances in Neural Information Processing Systems , 35:14274–14289, 2022

    2022

  29. [37]

    Learn” no” to say” yes” better: Improving vision- language models via negations

    Jaisidh Singh, Ishaan Shrivastava, Mayank Vatsa, Richa Singh, and Aparna Bharati. Learn” no” to say” yes” better: Improving vision- language models via negations. arXiv preprint arXiv:2403.20312, 2024

    2024 arXiv

  30. [38]

    Infants’ early understanding of different forms of negation

    Eszter Szab ´o and ´Agnes-Melinda Kov´acs. Infants’ early understanding of different forms of negation. 2022

    2022

  31. [39]

    Enhancing visual grounding in vision-language pre-training with position-guided text prompts

    Alex Jinpeng Wang, Pan Zhou, Mike Zheng Shou, and Shuicheng Yan. Enhancing visual grounding in vision-language pre-training with position-guided text prompts. IEEE Transactions on Pattern Analysis and Machine Intelligence , 46(5):3406–3421, 2023. JOURNAL OF LATEX CLASS FILES, ...

    2023

  32. [40]

    Tent: Fully test-time adaptation by entropy minimization

    Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: Fully test-time adaptation by entropy minimization. arXiv preprint arXiv:2006.10726 , 2020

    2006 arXiv

  33. [41]

    Tent: Fully test-time adaptation by entropy minimization

    Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: Fully test-time adaptation by entropy minimization. In International Conference on Learning Representations , 2021

    2021

  34. [42]

    Sun database: Large-scale scene recognition from abbey to zoo

    Jianxiong Xiao, James Hays, Krista A Ehinger, Aude Oliva, and Antonio Torralba. Sun database: Large-scale scene recognition from abbey to zoo. In 2010 IEEE computer society conference on computer vision and pattern recognition , pages 3485–3492. IEEE, 2010

    2010

  35. [43]

    Any-shift prompting for generalization over distributions

    Zehao Xiao, Jiayi Shen, Mohammad Mahdi Derakhshani, Shengcai Liao, and Cees GM Snoek. Any-shift prompting for generalization over distributions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 13849–13860, 2024

    2024

  36. [44]

    Msr-vtt: A large video description dataset for bridging video and language

    Jun Xu, Tao Mei, Ting Yao, and Yong Rui. Msr-vtt: A large video description dataset for bridging video and language. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 5288–5296, 2016

    2016

  37. [45]

    Test-time adaptation against multi-modal reliability bias

    Mouxing Yang, Yunfan Li, Changqing Zhang, Peng Hu, and Xi Peng. Test-time adaptation against multi-modal reliability bias. In The Twelfth International Conference on Learning Representations , 2024

    2024

  38. [46]

    C-tpt: Calibrated test-time prompt tuning for vision-language models via text feature dispersion

    Hee Suk Yoon, Eunseop Yoon, Joshua Tian Jin Tee, Mark Hasegawa- Johnson, Yingzhen Li, and Chang D Yoo. C-tpt: Calibrated test-time prompt tuning for vision-language models via text feature dispersion. In 12th International Conference on Learning Representations, ICLR 2024, 2024

    2024

  39. [47]

    When and why vision-language models be- have like bags-of-words, and what to do about it? arXiv preprint arXiv:2210.01936, 2022

    Mert Yuksekgonul, Federico Bianchi, Pratyusha Kalluri, Dan Juraf- sky, and James Zou. When and why vision-language models be- have like bags-of-words, and what to do about it? arXiv preprint arXiv:2210.01936, 2022

    2022 arXiv

  40. [48]

    Dual pro- totype evolving for test-time generalization of vision-language models

    Ce Zhang, Simon Stepputtis, Katia Sycara, and Yaqi Xie. Dual pro- totype evolving for test-time generalization of vision-language models. Advances in Neural Information Processing Systems , 37:32111–32136, 2024

    2024

  41. [49]

    Vision- language models for vision tasks: A survey

    Jingyi Zhang, Jiaxing Huang, Sheng Jin, and Shijian Lu. Vision- language models for vision tasks: A survey. IEEE transactions on pattern analysis and machine intelligence , 46(8):5625–5644, 2024

    2024

  42. [50]

    Test-time adaptation with clip reward for zero-shot generalization in vision-language models

    S Zhao, X Wang, L Zhu, and Y Yang. Test-time adaptation with clip reward for zero-shot generalization in vision-language models. In 12th International Conference on Learning Representations, ICLR 2024, 2024

    2024

  43. [51]

    Calling a spade a heart: Gaslighting multimodal large language models via negation

    Bin Zhu, Huiyan Qi, Yinxuan Gui, Jingjing Chen, Chong-Wah Ngo, and Ee-Peng Lim. Calling a spade a heart: Gaslighting multimodal large language models via negation. arXiv preprint arXiv:2501.19017 , 2025

    2025

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.