MINT: Memory-Infused Prompt Tuning at Test-time for CLIP

Jiaming Yi; Jishen Yang; Ruirui Pan; Xiulong Yang

REVIEW 4 major objections 6 minor 1 cited by

MINT: Memory-Infused Prompt Tuning at Test-time for CLIP

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read MINT claims that equipping CLIP with a learnable memory bank of visual prompts, assembled per image from hierarchical encoder features, improves test-time out-of-distribution accuracy beyond static prompt tuning.

desk verdict Incremental but sensible test-time prompt-tuning idea whose reported 'significant' gain is not supported by single-run numbers and test-set hyperparameter tuning. read the letter →

arxiv 2506.03190 v1 pith:KGN7IACX submitted 2025-05-31 cs.CV cs.AI

Jiaming Yi , Ruirui Pan , Jishen Yang , Xiulong Yang This is my paper

classification cs.CVcs.AI

keywords test-timeadaptationprompttuningCLIPmemorybankout-of-distributiongeneralizationassociativevisualpromptsentropyminimization

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

MINT claims that CLIP's out-of-distribution accuracy at test time can be improved by giving the image encoder a memory bank of reusable visual prompt components, assembled per image rather than using one static prompt. The method stores learnable key-value pairs, retrieves the most similar values using visual features from several encoder layers, and injects the averaged result into the first layer of the image encoder while also tuning text prompts. Reported results on four ImageNet variants put MINT's average Top-1 accuracy at 63.12 percent, above prior test-time adaptation and few-shot prompt methods. If true, this would mean a pre-trained vision-language model can adapt to shifting test distributions using only unlabeled data and a small, continuously updated prompt memory.

What carries the argument

The central object is the Memory Prompt Bank (MPB), a collection of learnable key-value entries $\{(k_i, v_i)\}$ where each key $k_i$ lives in image-token space and each value $v_i$ is a short sequence of $L_m$ visual prompt tokens. The mechanism works by treating layerwise [CLS] tokens of the ViT image encoder as queries, retrieving the top-$N_{\text{sel}}$ entries by cosine similarity at each of several layers, averaging the retrieved value prompts into one Associative Prompt $P_a$, and prepending that prompt to the patch embeddings at the first encoder layer. The bank and learnable text prompts are the only parameters updated; the pre-trained CLIP weights stay frozen. The retrieval-plus-composition step is what turns a fixed prompt into a sample-specific, memory-guided one.

What would settle it

Train MINT on one benchmark's test stream (for example ImageNet-R), freeze the Memory Prompt Bank, and run it on a different stream (for example ImageNet-Sketch); if accuracy falls back to the no-memory baseline, the bank has not acquired transferable visual semantics. A complementary check is to cluster the retrieved keys by ground-truth class or domain: if retrieved prompts for the same class are no more similar than for different classes, the associative memory is not encoding class-relevant visual context.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that a Memory Prompt Bank -- a set of $N_{\text{MPB}}=512$ learnable key-value pairs, each value a two-token visual prompt -- can serve as an associative memory for CLIP during online test-time adaptation. For each test image, MINT extracts query features from multiple layers of the image encoder, selects the top three keys by cosine similarity per layer, averages the corresponding value prompts into an Associative Prompt, and prepends it to the patch tokens at the input of the first Transformer block. The bank and a learnable text prompt are updated by minimizing prediction entropy on confident augmented views plus a term that rewards retrieval similarity. The paper reports 63.12 percent average Top-1 accuracy across ImageNet-R, ImageNet-A, ImageNet-V2, and ImageNet-Sketch, the best average among the compared zero-shot, few-shot, and test-time adaptation methods, with the largest single-dataset gain on ImageNet-R (78.68 percent).

Load-bearing premise

The load-bearing premise is that the key-value prompts, initialized from random noise and updated only by the entropy-plus-similarity loss on augmented test views, learn reusable visual semantics rather than patterns that happen to fit the augmentation noise of the current stream.

Editorial extensions

If this is right

CLIP can adapt to shifted test distributions without source data or labels by accumulating a small prompt memory from the test stream itself.
Dynamic retrieval from a bank outperforms a single general visual prompt, since the full configuration beats the fixed visual prompt version in the ablation on ImageNet-A.
The pre-trained weights remain frozen, so the adaptation cost is confined to the prompt parameters and the memory bank.
Bank capacity is a real hyperparameter: 512 entries and two tokens per memory prompt give the best ImageNet-A accuracy in the ablations, so the method's success depends on sizing the memory to the test stream's diversity.
The gains concentrate on render-style shifts such as ImageNet-R, where MINT reaches 78.68 percent, suggesting the memory mechanism helps most when the shift is stylistic rather than adversarial.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A direct but untested extension is that a bank trained on one distribution shift could be carried into another; if the retrieved prompts are semantically meaningful, freezing the bank after one benchmark and reusing it on another should retain part of the gain.
The loss couples entropy minimization with a similarity reward; varying $\lambda$ would reveal how much of the gain comes from memory semantics rather than from entropy regularization against prompt collapse.
The average lead over ZERO comes with a per-dataset trade-off: MINT is stronger on ImageNet-R and ImageNet-V2 but slightly weaker on ImageNet-A and ImageNet-Sketch, so the memory mechanism is not uniformly better across every shift type.
The design suggests a general recipe for other frozen encoders, since any model with layerwise token features could host a key-value prompt memory; the idea may transfer beyond CLIP to video or multi-modal encoders.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick read: MINT is a reasonable incremental step in test-time prompt tuning for CLIP. The core idea is a learnable key-value memory bank whose entries are retrieved by hierarchical image features and composed into an associative prompt injected into the image encoder, with text prompts updated alongside. That is a genuine extension of TPT/VPA-style adaptation, and the paper does the obvious ablations: removing components, varying bank size, prompt length, and injection layer. Code is promised. Credit where due: the method is specified well enough to reimplement, and the design story—memory of test-time patterns reused across samples—is coherent.

The soft spot is the evidence, not the idea. The headline claim is "top rank and significantly outperforming all compared models," but Table 1 reports single runs with no variance. The average margin over ZERO is 0.36 points, and MINT is actually lower on ImageNet-A and ImageNet-Sketch. With N(0,1) initialization and 64 random resized crops per image, seed-to-seed variation alone could plausibly cover that margin. So "significantly" is not supported. The paper also tunes NMPB, Lm, and injection layer on ImageNet-A (Fig. 3, Table 2) and then reports ImageNet-A as one of the four test benchmarks; that makes ImageNet-A not an independent test for MINT. The limitations paragraph concedes the MPB may need scenario-specific tuning, which further undercuts the generality claim. A smaller issue: Eq. (6) hand-waves dimension alignment across layers, and how Nlayers interacts with the single-injection claim is not fully pinned down.

None of this is fatal. The average gain may be real—MINT does improve on ImageNet-R and V2 by modest margins—but the paper needs repeated-seed results, paired statistics, and a cleaner protocol where hyperparameters are not chosen on a test set. As is, it is an honest incremental contribution that would benefit from major revision rather than acceptance as-is. I would send it to review; a serious referee could push the authors to make the empirical support match the language. I probably would not cite it yet, but I would keep an eye on the revision.

Referee Report

4 major / 6 minor

Summary. The paper proposes MINT, a test-time prompt tuning method for CLIP. It maintains a Memory Prompt Bank (MPB) of learnable key-value pairs; hierarchical visual features from multiple encoder layers retrieve the top-Nsel entries by cosine similarity, aggregate them into an Associative Prompt that is prepended to the first layer of the image encoder, and jointly update text prompts and bank entries using an entropy-minimization loss with a similarity regularizer. Experiments on ImageNet-R, ImageNet-A, ImageNet-V2, and ImageNet-Sketch report an average Top-1 accuracy of 63.12%, which the paper claims is the top rank and significantly better than all compared methods. The paper also includes component and hyperparameter ablations on ImageNet-A.

Significance. The memory-bank idea is a reasonable and timely extension of test-time prompt tuning, and the paper is clearly written with standard benchmarks, standard baselines, and released code. If the empirical gain is real, MINT would be a useful contribution to TTA for vision-language models. However, the central quantitative claim is currently supported only by single-run numbers: the average margin over the strongest baseline ZERO is 0.36 percentage points, MINT is worse than ZERO on ImageNet-A and ImageNet-Sketch, and the key hyperparameters were selected on ImageNet-A, which is also an evaluation benchmark. The associative-memory interpretation is also asserted rather than demonstrated. The strengths are the clarity of the method description and the inclusion of a limitations statement, but the evidence as presented is not yet sufficient for the paper's headline claims.

major comments (4)

[§4.4, Table 1] The central claim that MINT "significantly outperforms all compared models" is not supported by the reported numbers. Table 1 shows a single run per method with no standard deviation, no number of seeds, and no significance test. The average margin over ZERO is only 0.36 pp (63.12 vs 62.76), and MINT is actually lower than ZERO on ImageNet-A (59.83 vs 61.35) and ImageNet-Sketch (48.16 vs 48.29). Since the memory bank entries are initialized from N(0,1) and each test image generates 64 stochastic augmented views, seed-to-seed variance is expected; repeated runs or paired statistics are needed before the top-rank claim can be accepted.
[§4.5, Fig. 3] The hyperparameters NMPB=512, Lm=2, and first-layer injection are selected by ablations on ImageNet-A, and the same ImageNet-A results are then included in Table 1 and in the reported average. This makes ImageNet-A a test-set-tuned benchmark rather than an independent evaluation, and it also weakens the overall average. The authors should either choose hyperparameters on a separate validation split or show that the ranking across baselines is stable over a reasonable range of these hyperparameters.
[§3.2, Eq. (6)] The prompt composition operation is under-specified. Each v_j is defined as an element of R^{Lm×DI}, so a simple average of selected prompts is again in R^{Lm×DI}; however the text immediately adds that "averaging is performed if all selected Memory Prompts v_m share compatible dimensions; otherwise, alignment or projection is used." If dimensions can differ, Eq. (6) is not well-defined. Please state precisely how prompts from different layers are combined, whether the same bank is shared across layers, and what projection is used when dimensions are not compatible.
[§3.2 and §5 (Limitations)] The paper claims that the bank "works as a memory of previously seen samples" and that retrieved prompts carry semantically meaningful visual context, but no evidence is provided that the bank entries encode visual semantics rather than artifacts of random initialization or the similarity objective. Given the paper's own admission that MPB initialization and hyperparameters may require scenario-specific tuning, at least one diagnostic is needed—for example, retrieval patterns by domain or class, transfer of bank entries between datasets, or a simple probe of what individual memory prompts activate—to support the associative-memory interpretation.

minor comments (6)

[Fig. 3] The axis labels in Fig. 3 appear garbled (e.g., "1 282 565 121 0242 04859.05" and "0.0Accuracy(%)H yperparameter"), making the ablation plots difficult to read; please regenerate the figure with proper tick labels.
[Table 2] The checkmark columns in Table 2 are misaligned, and it is not immediately clear which configuration corresponds to each row; please use explicit ✓/✗ symbols with aligned rows.
[§3.1, Eq. (2)] The notation for the prompt sequence is inconsistent: the text defines P_l = {p_1,...,p_L} but Eq. (2) uses P_l as a single variable; please unify the notation.
[§4.3] The paper says "The aggregated Associative Prompt Pa is then added to the first layer of the image encoder," whereas §3.2 says the prompt is prepended to the patch sequence. Please clarify whether the operation is concatenation or additive injection.
[References] Reference [4] (ZERO) lacks venue information; please complete the bibliographic entry.
[§4.3] The computational overhead of MINT is not quantified, although the Limitations section mentions it; reporting wall-clock time or relative FLOPs versus TPT and ZERO would help readers assess the practical cost.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper reports an empirical test-time adaptation system evaluated on external benchmarks, with no derivation step that reduces to its own inputs.

full rationale

The paper's central claim is an empirical accuracy comparison (Table 1) between MINT and existing baselines on four ImageNet variants. There is no derivation chain in which a fitted parameter is renamed as a prediction: the Memory Prompt Bank is initialized from N(0,1), updated online via entropy minimization plus a cosine-similarity regularizer (Eq. 7), and evaluated on held-out benchmark test sets. Retrieval by cosine similarity between hierarchical image features and bank keys is an architectural mechanism, not a self-definitional definition of the reported accuracy. The loss does not contain the benchmark labels or the final Top-1 numbers, so the outcome is not forced by construction. Hyperparameters such as NMPB=512, Lm=2, and first-layer injection are selected via ablations on ImageNet-A, and the paper's Limitations section concedes that MPB initialization and hyperparameters may require scenario-specific tuning; this is an evaluation-integrity or soundness concern rather than a circularity of the derived claim. The paper invokes no uniqueness theorem and no load-bearing self-citation. The method is an empirical combination of known prompt-tuning and test-time adaptation components, and any weakness in the associative-memory semantics is a correctness/interpretability issue, not a circular reduction.

Assumptions & free parameters 9 free parameters · 4 assumptions · 2 invented entities

The central claim rests on the assumptions that test-time entropy minimization can train a randomly initialized key-value prompt bank, that cosine retrieval with CLIP features selects semantically meaningful prompts, and that prepending extra tokens to a ViT input is a harmless injection. Hyperparameters (bank size, prompt length, retrieval count, lambda, learning rate, confidence fraction) are free choices tuned on the evaluation data, and Nlayers is left unspecified.

free parameters (9)

lambda = 0.2
Balances entropy loss and retrieval-similarity term in Eq. (7); set by hand and explored in ablations.
NMPB = 512
Number of key-value pairs in the Memory Prompt Bank; selected by ablation on ImageNet-A (Fig. 3).
Lm = 2
Number of tokens per memory prompt; selected by ablation on ImageNet-A.
Nsel = 3
Number of memory prompts retrieved per query layer; set in implementation details.
B = 64
Number of augmented views per test image; used for confidence selection and loss.
kappa = 10%
Top-confidence quantile selected for gradient updates.
learning_rate = 5e-3
AdamW learning rate; optimization hyperparameter.
injection_layer = first layer
The associative prompt is prepended at the input to the first Transformer block; chosen via ablation (Fig. 3).
Nlayers
Number of hierarchical query layers used for retrieval is never specified in Section 4.3, making the method underspecified.

assumptions (4)

domain assumption CLIP frozen encoders provide useful hierarchical visual semantics at multiple layers
Section 3.2 extracts queries from Nlayers layers and assumes each level captures distinct visual abstraction usable for retrieval.
domain assumption Entropy minimization on confident augmented views is a valid label-free training signal for the prompt bank
Eq. (7) optimizes all learnable prompts with self-entropy, which is a standard TTA assumption but unproven for memory prompts.
ad hoc to paper Cosine similarity between query features and randomly initialized keys retrieves semantically relevant prompts after optimization
There is no guarantee or external validation that the key-value pairing corresponds to visual semantics rather than optimization artifacts.
ad hoc to paper Prepending extra prompt tokens to the ViT input does not disrupt pretrained representations or require positional-encoding changes
Section 3.2 specifies [Pa; Epatch(x)] as input to the first Transformer block without addressing positional embeddings or sequence-length consistency.

invented entities (2)

Memory Prompt Bank (MPB)
purpose: Stores learnable key-value prompt components that can be retrieved and composed at test time
No external falsifiable prediction; its utility is only visible through benchmark accuracy.
Associative Prompt Pa
purpose: Composite visual prompt injected into the image encoder to provide per-sample context
Constructed by averaging retrieved memory prompts; no independent evidence outside this paper's benchmarks.

how reviews work

0 comments

Cite this review

Pith. "Pith review of MINT: Memory-Infused Prompt Tuning at Test-time for CLIP." pith.science (2026). https://pith.science/paper/KGN7IACX

@misc{pith2026250603190,
  author       = {Pith},
  title        = {Pith review of: MINT: Memory-Infused Prompt Tuning at Test-time for CLIP},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KGN7IACX}},
  note         = {Machine review of arXiv:2506.03190}
}

read the original abstract

Improving the generalization ability of Vision-Language Pre-trained Models (VLMs) under test-time data distribution shifts remains a critical challenge. The existing Test-Time Adaptation (TTA) methods fall short in fully leveraging the model's internal knowledge, particularly in dynamically adapting to complex and hierarchical visual semantic information. In this paper, we propose Memory-Infused Prompt Tuning (MINT), a novel framework to address this issue. Inspired by human associative memory theory, MINT introduces a Memory Prompt Bank (MPB), which stores learnable key-value prompt pairs that work as a memory of previously seen samples. During the test time, relevant prompt pairs in the MPB are retrieved by the hierarchical visual features of test images to dynamically assemble Associative Prompts. The associative prompts are then injected into the image encoder for fine-grained, customized visual contextual guidance. MINT also utilizes learnable text prompts. MINT thus enables rapid, precise VLM adaptation at test time by leveraging this MPB-acquired memory, without source data or retraining. The code is available at https://github.com/Jamieyi2004/MINT.

Figures

Figures reproduced from arXiv: 2506.03190 by the authors.

**Figure 1.** Distribution of test-time samples on ImageNet-R. Different classes present distinct domain shifts, motivating adaptive strategies for TTA. photo of a [class]”) for downstream tasks. Although practically applicable, manual prompt design is time-consuming, requires expertise, and lacks generality. To overcome these limitations, subsequent research introduced prompt tuning, treating prompts as learnable parameters op… view at source ↗

**Figure 2.** Overview of our proposed Memory-Infused Prompt Tuning (MINT) framework at Test-time for CLIP. 3.1 Preliminaries: CLIP and Prompt Tuning A pre-trained CLIP model, denoted as θ = {EI , ET }, comprises an image encoder EI and a text encoder ET . These encoders project images and text descriptions into a shared multimodal embedding space. Typically, EI is a Vision Transformer (ViT) [3] or a Convolutional Neural Networ… view at source ↗

**Figure 3.** Ablation study about the effect of different hyper-parameters on ImageNet-A. MPB to dynamically generate and integrate customized Associative Prompts injected into the image encoder for fine-grained visual guidance, synergizing with learnable text prompts. Experiments show MINT significantly enhances CLIP’s OOD generalization and robustness. This stems from its leveraging not only internal pre-trained knowledge bu… view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Adapting Vision-Language Models Without Labels: A Comprehensive Survey
cs.LG 2025-08 conditional novelty 5.0 of 10

A survey that organizes unsupervised vision-language model adaptation by unlabeled-data availability into four paradigms: data-free transfer, domain transfer, episodic test-time, and online test-time adaptation.

Reference graph

Works this paper leans on

29 extracted references · 19 canonical work pages · cited by 1 Pith paper

[1]

Cai, Z., Kwon, G., Ravichandran, A., Bas, E., Tu, Z., Bhotika, R., Soatto, S.: X-detr: A versatile architecture for instance-wise vision-language tasks (2022)

work page 2022
[2]

In: 2009 IEEE conference on computer vision and pattern recognition

Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: A large- scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition. pp. 248–255. Ieee (2009)

2009
[3]

In: International Con- ference on Learning Representations (2021)

Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al.: An image is worth 16x16 words: Transformers for image recognition at scale. In: International Con- ference on Learning Representations (2021)

2021
[4]

Farina, M., Franchi, G., Iacca, G., Mancini, M., Ricci, E.: Frustratingly easy test- time adaptation of vision-language models (2024)

work page 2024
[5]

He,K.,Zhang,X.,Ren,S.,Sun,J.:Deepresiduallearningforimagerecognition.In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 770–778 (2016)

2016
[6]

In: Proceedings of the IEEE/CVF International Conference on computer vision

Hendrycks, D., Basart, S., Mu, N., Kadavath, S., Wang, F., Dorundo, E., Desai, R., Zhu, T., Parajuli, S., Guo, M., et al.: The many faces of robustness: A critical MINT: Memory-Infused Prompt Tuning at Test-time for CLIP 13 analysis of out-of-distribution generalization. In: Proceedings of the IEEE/CVF International Conference on computer vision. pp. 8340...

work page 2021
[7]

In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

Hendrycks, D., Zhao, K., Basart, S., Steinhardt, J., Song, D.: Natural adversarial examples. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 15262–15271 (2021)

2021
[8]

In: International conference on machine learning

Jia, C., Yang, Y., Xia, Y., Chen, Y.T., Parekh, Z., Pham, H., Le, Q., Sung, Y.H., Li, Z., Duerig, T.: Scaling up visual and vision-language representation learning with noisy text supervision. In: International conference on machine learning. pp. 4904–4916. PMLR (2021)

2021

Show all 29 references

[9]

In: Computer Vision – ECCV 2022

Jia, M., Tang, L., Chen, B.C., Cardie, C., Belongie, S., Hariharan, B., Lim, S.N.: Visual prompt tuning. In: Computer Vision – ECCV 2022. pp. 709–727. Springer Nature Switzerland, Cham (2022)

2022
[10]

In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)

Kamath, A., Singh, M., LeCun, Y., Synnaeve, G., Misra, I., Carion, N.: Mdetr - modulated detection for end-to-end multi-modal understanding. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). pp. 1780– 1790 (October 2021)

2021
[11]

In: Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition (CVPR)

Khattak, M.U., Rasheed, H., Maaz, M., Khan, S., Khan, F.S.: Maple: Multi-modal prompt learning. In: Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition (CVPR). pp. 19113–19122 (June 2023)

2023
[12]

In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (2021)

Lester, B., Al-Rfou, R., Constant, N.: The power of scale for parameter-efficient prompt tuning. In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (2021)

2021
[13]

In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Li, L.H., Zhang, P., Zhang, H., Yang, J., Li, C., Zhong, Y., Wang, L., Yuan, L., Zhang, L., Hwang, J.N., Chang, K.W., Gao, J.: Grounded language-image pre- training. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 10965–10975 (...

2022
[14]

In: Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., Levine, S

Liu, H., Li, C., Wu, Q., Lee, Y.J.: Visual instruction tuning. In: Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., Levine, S. (eds.) Advances in Neural Information Processing Systems. pp. 34892–34916. Curran Associates, Inc. (2023)

2023
[15]

Liu, T., Wu, Z., Xiong, W., Chen, J., Jiang, Y.G.: Unified multimodal pre-training and prompt-based tuning for vision-language understanding and generation (2021)

2021
[16]

Niu, S., Wu, J., Zhang, Y., Wen, Z., Chen, Y., Zhao, P., Tan, M.: Towards stable test-time adaptation in dynamic wild world (2023)

2023
[17]

In: Proceedings of the 38th International Conference on Machine Learning (2021)

Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., Sutskever, I.: Learning transfer- able visual models from natural language supervision. In: Proceedings of the 38th International Conference on M...

2021
[18]

arXiv preprint arXiv:2204.06125 1(2), 3 (2022)

Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., Chen, M.: Hierarchical text- conditional image generation with clip latents. arXiv preprint arXiv:2204.06125 1(2), 3 (2022)

2022 arXiv
[19]

Recht, B., Roelofs, R., Schmidt, L., Shankar, V.: Do imagenet classifiers generalize to imagenet? In: International conference on machine learning. pp. 5389–5400. PMLR (2019)

2019
[20]

In: Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., Oh, A

Shu, M., Nie, W., Huang, D.A., Yu, Z., Goldstein, T., Anandkumar, A., Xiao, C.: Test-time prompt tuning for zero-shot generalization in vision-language models. In: Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., Oh, A. (eds.) Advances in Neural Information Process...

2022
[21]

In: Proceedings of the 31st ACM International Conference on Multimedia

Sun, J., Ibrahim, M., Hall, M., Evtimov, I., Mao, Z.M., Ferrer, C.C., Hazirbas, C.: Vpa: Fully test-time visual prompt adaptation. In: Proceedings of the 31st ACM International Conference on Multimedia. pp. 5796–5806 (2023) 14 J. Yi et al

2023
[22]

Advances in neural information pro- cessing systems 30 (2017)

Vaswani,A.,Shazeer,N.,Parmar,N.,Uszkoreit,J.,Jones,L.,Gomez,A.N.,Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information pro- cessing systems 30 (2017)

2017
[23]

Wang, D., Shelhamer, E., Liu, S., Olshausen, B., Darrell, T.: Tent: Fully test-time adaptation by entropy minimization (2021)

2021
[24]

Advances in neural information processing systems 32 (2019)

Wang, H., Ge, S., Lipton, Z., Xing, E.P.: Learning robust global representations by penalizing local predictive power. Advances in neural information processing systems 32 (2019)

2019
[25]

In: ECCV (2022)

Xu, M., Zhang, Z., Wei, F., Lin, Y., Cao, Y., Hu, H., Bai, X.: A simple baseline for open-vocabulary semantic segmentation with pre-trained vision-language model. In: ECCV (2022)

2022
[26]

Zanella, M., Ben Ayed, I.: On the test-time zero-shot generalization of vision- language models: Do we really need prompt learning? In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 23783–23793 (June 2024)

2024
[27]

Advances in neural information processing systems35, 38629–38642 (2022)

Zhang, M., Levine, S., Finn, C.: Memo: Test time robustness via adaptation and augmentation. Advances in neural information processing systems35, 38629–38642 (2022)

2022
[28]

In: Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition (CVPR)

Zhou, K., Yang, J., Loy, C.C., Liu, Z.: Conditional prompt learning for vision- language models. In: Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition (CVPR). pp. 16816–16825 (June 2022)

2022
[29]

International Journal of Computer Vision (9), 2337–2348 (Jul 2022)

Zhou, K., Yang, J., Loy, C.C., Liu, Z.: Learning to prompt for vision-language models. International Journal of Computer Vision (9), 2337–2348 (Jul 2022)

2022

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Cai, Z., Kwon, G., Ravichandran, A., Bas, E., Tu, Z., Bhotika, R., Soatto, S.: X-detr: A versatile architecture for instance-wise vision-language tasks (2022)

work page 2022

[2] [2]

In: 2009 IEEE conference on computer vision and pattern recognition

Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: A large- scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition. pp. 248–255. Ieee (2009)

2009

[3] [3]

In: International Con- ference on Learning Representations (2021)

Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al.: An image is worth 16x16 words: Transformers for image recognition at scale. In: International Con- ference on Learning Representations (2021)

2021

[4] [4]

Farina, M., Franchi, G., Iacca, G., Mancini, M., Ricci, E.: Frustratingly easy test- time adaptation of vision-language models (2024)

work page 2024

[5] [5]

He,K.,Zhang,X.,Ren,S.,Sun,J.:Deepresiduallearningforimagerecognition.In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 770–778 (2016)

2016

[6] [6]

In: Proceedings of the IEEE/CVF International Conference on computer vision

Hendrycks, D., Basart, S., Mu, N., Kadavath, S., Wang, F., Dorundo, E., Desai, R., Zhu, T., Parajuli, S., Guo, M., et al.: The many faces of robustness: A critical MINT: Memory-Infused Prompt Tuning at Test-time for CLIP 13 analysis of out-of-distribution generalization. In: Proceedings of the IEEE/CVF International Conference on computer vision. pp. 8340...

work page 2021

[7] [7]

In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

Hendrycks, D., Zhao, K., Basart, S., Steinhardt, J., Song, D.: Natural adversarial examples. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 15262–15271 (2021)

2021

[8] [8]

In: International conference on machine learning

Jia, C., Yang, Y., Xia, Y., Chen, Y.T., Parekh, Z., Pham, H., Le, Q., Sung, Y.H., Li, Z., Duerig, T.: Scaling up visual and vision-language representation learning with noisy text supervision. In: International conference on machine learning. pp. 4904–4916. PMLR (2021)

2021