Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

Prompting without Panic: Attribute-aware, Zero-shot, Test-Time Calibration

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Frozen LLM attributes and a text-dispersion loss fix the overconfidence that test-time prompt tuning causes in CLIP, cutting average calibration error from 11.7 to 4.11.

desk verdict A useful calibration technique with an overclaimed zero-shot headline; the attribute-initialization idea deserves a serious look. read the letter →

arxiv 2506.22819 v1 pith:PUKBNUBC submitted 2025-06-28 cs.CV cs.LG

Ramya Hebbalaguppe , Tamoghno Kandar , Abhinav Nagpal , Chetan Arora This is my paper
classification cs.CVcs.LG
keywords test-timeprompttuningconfidencecalibrationexpectederrorvision-languagemodelsCLIPLLMattributegroundingtextfeaturedispersionzero-shotclassification
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Test-time prompt tuning lets a CLIP model adapt to a single test image, but the adaptation makes the model overconfident: the entropy-minimizing objective that drives TPT pushes predicted probabilities toward 0 or 1. The paper argues that this miscalibration is an overfitting problem caused by random or naive prompt initialization, and that it can be avoided without labels. The proposed method, Test-time Calibration via Attribute Alignment (TCA), initializes the tunable prompt with frozen, relevance-ranked attributes for each class generated by a large language model, and adds intra-class contraction plus inter-class separation losses over text embeddings. On 11 fine-grained datasets it reports an average expected calibration error (ECE) of 4.11, compared with 11.7 for vanilla TPT and 6.12 for C-TPT, while keeping accuracy roughly unchanged. If the claim is right, test-time adaptation for zero-shot vision-language models does not have to sacrifice calibrated confidence.

What carries the argument

The load-bearing object is the attribute-anchored text prompt $p \oplus a \oplus c$: a learnable prefix $p$, a frozen attribute vector $a$ supplied by an LLM and ranked by cosine similarity to the class name, and the class name $c$. From these prompts, per-class text embeddings are averaged into centroids, and the two regularization terms operate on them. $L_{\mathrm{intra}}$ is the mean text attribute spread (MTAS), the average squared distance from attribute embeddings to their class centroid, minimized to contract each class; $L_{\mathrm{inter}}$ is the negative average text feature dispersion (ATFD), the squared distance between class centroids and their global mean, minimized to push classes apart. The full loss is $L_{\mathrm{total}} = L_{\mathrm{TPT}} + \alpha L_{\mathrm{inter}} + \beta L_{\mathrm{intra}}$ with $\alpha = 10$, $\beta = 35$ for fine-grained datasets, and the freeze on $a$ and $c$ is what prevents the updated prompt from memorizing a single test sample.

What would settle it

Recompute TCA's average ECE using only the ten datasets that were not used for hyperparameter selection, or tune $\alpha$ and $\beta$ on a held-out split: if that average rises toward C-TPT's 6.12 instead of staying near 4.11, the calibration gain is largely an artifact of tuning on a member of the benchmark.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central claim is that miscalibration after test-time prompt tuning is not a fixed property of the VLM but an artifact of how the prompt is initialized and updated. For each class, an LLM supplies descriptive attributes; the top two, ranked by cosine similarity to the class name, are frozen into the prompt $p \oplus a \oplus c$, where only the template tokens $p$ are learnable. The tuning objective adds to the standard TPT entropy loss two text-side regularizers: an intra-class term that pulls attribute embeddings of a class toward their centroid (mean text attribute spread) and an inter-class term that pushes class centroids away from the global mean (average text feature dispersion). The ablation on DTD attributes alone reduces ECE by 20.6%, the TCA loss alone reduces it by $3.65\times$, and both together reduce it by $4.59\times$ from 25.7 to 5.59, which the paper takes as evidence that both components carry the calibration gain.

Load-bearing premise

The reported 4.11 average assumes that the two loss weights ($\alpha = 10$, $\beta = 35$) chosen once on Caltech101, and the inclusion of Caltech in the averaged benchmark, transfer to every other dataset and distribution shift.

Editorial extensions

If this is right

  • The recipe is a plug-in: adding TCA on top of PromptAlign brings average ECE on the ViT-B/16 fine-grained suite from 10.04 down to 4.58, and adding it to DiffTPT brings average ECE from 6.78 to 5.72 (RN50) and from 9.68 to 4.99 (ViT-B/16).
  • The calibration improvement is not bought with accuracy: on the 11 fine-grained datasets TCA holds accuracy essentially at the C-TPT level (57.85 vs 57.8 for RN50, 63.91 vs 64.8 for ViT-B/16).
  • Both the ablations and the t-SNE plots support the paper's explanation: frozen attribute tokens lower ECE by themselves, and the full text-dispersion loss produces the most separated class embeddings, which the paper reads as the mechanism behind the calibration gain.
  • On natural distribution shifts the benefit partially transfers: TCA beats C-TPT in ECE on 3 of the 4 ImageNet variants in the reported configurations.
  • Since only template tokens are updated and both encoders stay frozen, the method adds no training data, no label supervision, and no model fine-tuning, so it remains a zero-shot, test-time procedure.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: because $\alpha$ and $\beta$ were grid-searched on Caltech101 and Caltech is included in the reported 11-dataset average, the headline 4.11 could be partly in-sample; re-evaluating with the ten remaining datasets or with per-dataset weights would reveal how much of the gain actually transfers.
  • Editorial inference: the same frozen-attribute plus text-dispersion mechanism is not tied to CLIP or to the two baselines tested, so it could plausibly be dropped into other test-time and few-shot prompt tuners and other vision-language backbones.
  • Editorial inference: a natural testable upgrade is to draw attributes per test image or per augmented view rather than per class name, which would let the grounding adapt when class-level attributes are ambiguous.
  • Editorial inference: the text-dispersion regularizer might serve as a calibration-oriented objective beyond prompting, for example in continual or parameter-efficient fine-tuning of vision-language models, but the paper reports evidence only for the test-time prompt setting.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes TCA, a test-time prompt-tuning method for CLIP that (1) initializes tunable prompts with LLM-generated class attributes and (2) adds an inter-class dispersion loss and an intra-class contraction loss to the standard TPT entropy objective. The authors report average ECE of 4.11 on 11 fine-grained datasets for CLIP-RN50, compared with 11.7 for vanilla TPT and 6.12 for C-TPT, while keeping accuracy essentially unchanged, and they report gains on four ImageNet distribution-shift variants and on top of DiffTPT and PromptAlign. The central claim is that frozen attribute grounding and text-feature dispersion regularization jointly prevent the sample overfitting that causes miscalibration after TPT.

Significance. If the reported numbers hold, the contribution is useful: it is a simple, largely hyperparameter-light modification of test-time prompt tuning, it addresses an under-explored calibration problem, and it includes ablations, integration with existing prompt-tuning methods, and multiple architectures. The authors also make code available and conduct OOD evaluations. The empirical strength of the paper is, however, not yet established because the headline average includes the dataset used to select the loss weights and attribute count, and because one reported ViT-B/16 row appears to be a duplicate of the RN50 row. These issues are fixable but currently prevent the results from being read as a zero-shot transfer claim.

major comments (4)
  1. [§4, Implementation Details; Table 1] The reported 4.11 average is partially in-sample. The text says the grid search over (alpha, beta) was performed on Caltech101 for least ECE, and Caltech101 is one of the 11 datasets in the average. Likewise, the attribute count M=2 is selected after trying 1, 2, and 3 attributes, with the choice based on 'best ECE values on majority of datasets', i.e., on the same benchmark suite. Please report results excluding the tuning dataset from the average, or use a nested/leave-one-dataset-out selection protocol, and state the sensitivity of ECE to (alpha, beta, M). The same concern applies to the OOD numbers in Table 2, where no selection protocol is described for the (45, 15) weights.
  2. [Eq. (4), Eq. (7), Algorithm 1] The two regularization losses are not scale-normalized. L_intra is an average of squared L2 distances over M attribute embeddings, and L_inter is an average over K class means of squared distances to the global mean. With K varying from 10 (EuroSAT) to 1000 (ImageNet), and with attribute-embedding norms varying across classes and datasets, the fixed weights (alpha=10, beta=35) can have very different effective strengths. This makes the transfer of Caltech-tuned weights a load-bearing assumption. The paper should either normalize these losses (e.g., by dimension or by per-class averaging of already-normalized features) or provide a per-dataset sensitivity analysis showing that the chosen weights are not operating in very different regimes.
  3. [Table 1, CLIP-ViT-B/16 section] The row labeled '+TPTEnsemble+TCA (2 attributes)' under CLIP-ViT-B/16 is numerically identical to the '+TPTEnsemble+TCA (2 Attributes)' row under CLIP-RN50 (accuracy 68.1/93.2/90.13/65.94/68.9/84.23/25.38/65.84/43.91/47.17/67.72/65.50 and ECE 1.88/3.09/4.38/3.93/3.57/1.91/3.36/6.02/4.36/9.36/2.71/4.05). Since every other row in the table differs between the two backbones, this appears to be a transcription/duplication error. The claimed ViT-B/16 ensemble average ECE of 4.05 in §4.1 therefore cannot be verified, and all ViT-B/16 averages derived from this row need to be recomputed.
  4. [§3.3, §4.3, Fig. 4] The causal claim that greater text-feature dispersion is what produces lower ECE is supported only by qualitative t-SNE plots and by a DTD ablation. The paper correctly notes the inverse correlation, but it does not test whether the ECE improvement is driven by changed confidence values rather than by feature geometry. A simple quantitative check, such as computing ECE before and after freezing the prompt but applying the same attribute initialization, would strengthen the attribution. As written, the mechanism section is suggestive rather than demonstrated, and the incomplete sentences around Algorithm 1 lines 8, 9, and 12 make the intended mechanism hard to parse.
minor comments (6)
  1. [Abstract] The arXiv abstract states 'We make three contributions' but then lists only two numbered contributions; the full-text abstract says 'two contributions' while the Introduction lists three. Please make these consistent.
  2. [Eq. (9)] The display 'Ltotal = p* = arg min_p [...]' is malformed. It should be written as p* = arg min_p [L_TPT + alpha L_inter + beta L_intra], with L_total defined separately as the objective being minimized.
  3. [§3.3] Two sentences are incomplete: 'The use of the term (See Algorithm 1 line 12) plays a critical role...' and '(See Algorithm 1 lines 8 and 9) takes care of aligning similar textual embeddings.' The referents of 'the term' are missing, and the sentences should name the ATFD/MTAS quantities explicitly.
  4. [Related Works] There is a typo 'zero-short' for 'zero-shot' in the second paragraph of Section 2.
  5. [Table 1 caption and body] The caption says 'please see main test for configuration details' (should be 'main text'), and the ECE column for the RN50 TCA row is printed as '04.11' with a stray leading zero. Also, the table header uses inconsistent abbreviations ('Acc.' vs 'Accuracy') across Tables 1, 2, 4, 5, and 6.
  6. [Table 4 caption] The caption states that the first three row-pairs are 'borrowed from C-TPT paper', but this borrowing is not flagged in the main text and the source table is not cited precisely. Please add a citation and confirm those numbers match the original source.

Circularity Check

0 steps flagged · score 1.0 of 10

No circular derivation: TCA's attribute initialization and text-dispersion losses are defined independently of ECE and measured against external baselines; the only caveat is in-sample hyperparameter selection, which is a robustness concern, not circularity.

full rationale

The claimed derivation chain is not circular. TCA's two components are specified independently of the ECE metric: LLM-generated attributes are selected by cosine similarity to class names (Sec. 3.2), and the regularization loss combines L_TPT with L_inter = -ATFD, taken explicitly from the external C-TPT work, plus L_intra, an intra-class analogue of ATFD (Sec. 3.2, Eqs. 4-8). Neither loss re-encodes ECE, and no equation in the paper equates the reported ECE to the method's inputs. The headline 4.11 ECE is computed from the standard ECE definition on held-out test data and compared against external baselines (TPT, C-TPT, DiffTPT, PromptAlign), so the central claim is empirically falsifiable and not self-referential. The paper's self-citations [10,13,14,15,39,44] appear only as examples of train-time calibration methods in Related Work and do not justify the TCA design or its calibration improvement. The one flagged caveat is in Section 4 (Implementation Details): alpha and beta are grid-searched for least ECE on Caltech101, and the attribute count M=2 is selected via an ablation on DTD; because Caltech101 is included in the 11-dataset average, the 4.11 number is partly in-sample. This is a selection-bias/correctness limitation, not a circular reduction: the selected hyperparameters do not by construction determine the reported ECE, and the method is still evaluated on held-out datasets such as ImageNet, SUN397, and EuroSAT. Overall, no step reduces a prediction to a fitted input or a self-citation chain; the residual score of 1 reflects only non-load-bearing self-citations and the in-sample tuning caveat.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on three fitted quantities (alpha, beta, M) and four background assumptions. The most consequential is the transfer of a single hyperparameter pair and attribute count across heterogeneous datasets, especially because the headline average includes the dataset used for selection. No new physical or conceptual entities are introduced.

free parameters (3)
  • alpha (inter-class loss weight) = 10 (fine-grained), 45 (OOD)
    Grid-searched on Caltech101 to minimize ECE, then applied to all datasets. This is a data-dependent tuning choice that affects the balance between calibration losses and TPT loss.
  • beta (intra-class loss weight) = 35 (fine-grained), 15 (OOD)
    Grid-searched on Caltech101 together with alpha; same transfer assumption across all datasets.
  • M (number of attributes per class) = 2
    Selected because '2 attributes gave the best ECE values on majority of datasets' per Implementation Details; a post-hoc choice from the evaluation suite itself.
assumptions (4)
  • domain assumption CLIP text-embedding geometry is a valid proxy for predictive calibration: increasing inter-class text dispersion and reducing intra-class attribute spread lowers ECE.
    Invoked in Sec. 3.3 and Sec. 4.4; the paper relies on the C-TPT ATFD intuition and t-SNE plots, but provides no formal link between text dispersion and calibration error.
  • domain assumption LLM-generated attributes are relevant visual descriptors, and ranking them by cosine similarity to the class name in CLIP text space selects the most useful attributes.
    Used in Sec. 3.2 and Fig. 2 to build the frozen attribute prefix for every class; the paper does not evaluate attribute quality independently of final ECE.
  • domain assumption A single test image with augmentations and the TPT confidence-selection filter is a sufficient self-supervisory signal for prompt adaptation.
    Inherited from TPT [32] and used as the base loss L_TPT in Eq. 2; the paper relies on this to avoid labeled data.
  • ad hoc to paper One hyperparameter pair (alpha, beta) selected on Caltech101 transfers to all other datasets and to OOD shifts.
    Stated in Implementation Details: grid search on Caltech for least ECE, then same values applied to 11 datasets; Caltech is included in the reported average, so this is a load-bearing selection choice rather than a validated general rule.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Prompting without Panic: Attribute-aware, Zero-shot, Test-Time Calibration." pith.science (2026). https://pith.science/paper/PUKBNUBC

@misc{pith2026250622819,
  author       = {Pith},
  title        = {Pith review of: Prompting without Panic: Attribute-aware, Zero-shot, Test-Time Calibration},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PUKBNUBC}},
  note         = {Machine review of arXiv:2506.22819}
}
read the original abstract

Vision-language models (VLM) have demonstrated impressive performance in image recognition by leveraging self-supervised training on large datasets. Their performance can be further improved by adapting to the test sample using test-time prompt tuning (TPT). Unfortunately, the singular focus of TPT approaches on improving the accuracy suffers from tunnel vision, and leads to degradation in confidence calibration. This limits the applicability of TPT in critical applications. We make three contributions in this work. (1) We posit that random or naive initialization of prompts leads to overfitting on a particular test sample, and is the main reason for miscalibration of the VLM after TPT. To mitigate the problem, we propose careful initialization of test time prompt using prior knowledge about the target label attributes from a large language model (LLM); (2) To further maintain the quality of prompts during \tpt, we propose a novel regularization loss to reduce intraclass distance, and increase inter-class distance between the learnt Through extensive experiments on different CLIP architectures and 15 datasets, we show that our approach can effectively improve the calibration after TPT. We report an average expected calibration error (ECE) of 4.11 with our method, TCA, compared to 11.7 for vanilla TPT, 6.12 for C-TPT (ICLR'24), 6.78 for DiffTPT (CVPR'23), and 8.43 for PromptAlign (NeurIPS'23). The code is publicly accessible at: https://github.com/rhebbalaguppe/TCA_PromptWithoutPanic.

Figures

Figures reproduced from arXiv: 2506.22819 by the authors.

Figure 1
Figure 1. Conceptual comparison between our proposed TCA vs. the con￾temporaries. Test-time prompt tuning methods, such as TPT [32], learn test￾time prompts through parameter optimization. However, these methods often face performance disadvantages in calibration, as they struggle to dynamically adapt to varying textual feature distributions, limiting effective prompt cali￾bration. Methods, ArgGue[49], DiffTPT[9], and PromptA… view at source ↗
Figure 2
Figure 2. Visual attributes are extracted by prompting a multimodal foundational [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Calibration using Test-time Attribute Alignment for zero-shot image classification: In a typical test time prompt tuning for image classifi￾cation, a category label is prefixed with a template text, such as “a photo of a” (e.g., “a photo of a red panda”) to generate the prompt for tuning. Our approach differs in the following ways: (a) Visual attributes are extracted as shown in [PITH_FULL_IMAGE:figures/full_fig_p0… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: The t-SNE plot shows Class-specific Text Embeddings on tuned prompts. [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]
Figure 5
Figure 5. Figure 5: t-SNE visualization: Class-specific Text Embeddings are shown via t￾SNE for the tuned prompts on (a) Flowers102 [34], (b) OxfordPets [38] and (c) UCF101 [48] datasets. Each color in the figure denotes a unique prompt. We can see TCA exhibits the lowest ECE on Flower102…
Figure 6
Figure 6. Figure 6: The t-SNE plot of prompt visualizations for the proposed [PITH_FULL_IMAGE:figures/full_fig_p025_6.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. When Prompts Ignore Structure: Graph-Based Attribute Reasoning for Calibrated VLMs

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Selecting class attributes via a graph attention network instead of flat similarity ranking lowers average Expected Calibration Error in CLIP test-time prompt tuning.

Reference graph

Works this paper leans on

63 extracted references · 57 canonical work pages · cited by 1 Pith paper

  1. [1]

    In: ECCV (2014) 10, 23

    Bossard, L., Guillaumin, M., Gool, L.: Food-101:mining discriminative com- ponents with random forests. In: ECCV (2014) 10, 23

    work page 2014

  2. [2]

    In: ICLR (2023) 4, 5, 9

    Chen, G., Yao, W., Song, X., Li, X., Rao, Y., Zhang, K.: PLOT: Prompt learning with optimal transport for vision-language models. In: ICLR (2023) 4, 5, 9

    work page 2023

  3. [3]

    In: Proceedings of the IEEE/CVF ICCV

    Cho, E., Kim, J., Kim, H.J.: Distribution-aware prompt tuning for vision- language models. In: Proceedings of the IEEE/CVF ICCV. pp. 22004–22013 (2023) 14

    work page 2023

  4. [4]

    In: CVPR

    Cimpoi, M., Maji, S., Kokkinos, I., S., Vedaldi, A.: Describing textures in the wild. In: CVPR. pp. 3606–3613 (2014) 10, 23

    work page 2014

  5. [5]

    Journal of the American Statis- tical Association (1982) 5

    Dawid, A.P.: The well-calibrated bayesian. Journal of the American Statis- tical Association (1982) 5

    work page 1982

  6. [6]

    In: CVPR

    Deng, J., Dong, W., Socher, R., Li, L., Li, K., Fei-Fei, L.: Imagenet: A large- scale hierarchical image database. In: CVPR. pp. 248–255 (2009) 10, 22, 23

    work page 2009

  7. [7]

    arXiv preprint arXiv:2010.11929 (2020) 24

    Dosovitskiy, A.: An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020) 24

    arXiv 2020

  8. [8]

    In: CVPR Workshops

    Fei-Fei, L., Fergus, R., Perona, P.: Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories. In: CVPR Workshops. p. 178 (2004) 10

    work page 2004

Show all 63 references
  1. [9]

    ICCV (2023) 1, 2, 3, 4, 10, 14, 20, 26

    Feng, C.M., Yu, K., Liu, Y., Khan, S., Zuo, W.: Diverse data augmentation with diffusions for effective test-time prompt tuning. ICCV (2023) 1, 2, 3, 4, 10, 14, 20, 26

    2023

  2. [10]

    In: Joint European Confer- ence on Machine Learning and Knowledge Discovery in Databases

    Ghosal, S., Hebbalaguppe, R., Manocha, D.: Better features, better calibra- tion: A simple fix for overconfident networks. In: Joint European Confer- ence on Machine Learning and Knowledge Discovery in Databases. Springer (2024) 4

    2024

  3. [11]

    In: ICML

    Guo, C., Pleiss, G., Sun, Y., Weinberger, K.Q.: On calibration of modern neural networks. In: ICML. vol. 70, pp. 1321–1330 (2017) 2, 4

    2017

  4. [12]

    In: CVPR (2023) 5

    Hantao Yao, Rui Zhang, C.X.: Visual-language prompt tuning with knowledge-guided context optimization. In: CVPR (2023) 5

    2023

  5. [13]

    In: Proceedings of the Asian Conference on Computer Vision

    Hebbalaguppe, R., Baranwal, M., Anand, K., Arora, C.: Calibration trans- fer via knowledge distillation. In: Proceedings of the Asian Conference on Computer Vision. pp. 513–530 (2024) 4

    2024

  6. [14]

    In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases

    Hebbalaguppe, R., Ghosal, S.S., Prakash, J., Khadilkar, H., Arora, C.: A novel data augmentation technique for out-of-distribution sample detection using compounded corruptions. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases. pp. 529–545...

    2022

  7. [15]

    16 Hebbalaguppe et al

    Hebbalaguppe, R., Prakash, J., Madan, N., Arora, C.: A stitch in time saves nine: A train-time regularizing loss for improved neural network calibration. 16 Hebbalaguppe et al. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 16...

    2022

  8. [16]

    Helber, P., Bischke, B., Dengel, A., Borth, D.: Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE J. Sel. Top. Appl. Earth Obs. Remote. Sens.12(7), 2217–2226 (2019) 10, 23

    2019

  9. [17]

    ICCV (2021) 10, 22, 23

    Hendrycks, D., Basart, S., Mu, N., Kadavath, S., Wang, F., Dorundo, E., Desai, R., Zhu, T., Parajuli, S., Guo, M., Song, D., Steinhardt, J., Gilmer, J.: The many faces of robustness: A critical analysis of out-of-distribution generalization. ICCV (2021) 10, 22, 23

    2021

  10. [18]

    CVPR (2021) 10, 22, 23

    Hendrycks, D., Zhao, K., Basart, S., Steinhardt, J., Song, D.: Natural ad- versarial examples. CVPR (2021) 10, 22, 23

    2021

  11. [19]

    In: ECCV

    Jia, M., Tang, L., Chen, B.C., Cardie, C., Belongie, S., Hariharan, B., Lim, S.N.: Visual prompt tuning. In: ECCV. pp. 709–727 (2022) 5

    2022

  12. [20]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Karmanov, A., Guan, D., Lu, S., El Saddik, A., Xing, E.: Efficient test-time adaptation of vision-language models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 14162–14171 (2024) 3

    2024

  13. [21]

    NeurIPS 33, 18661–18673 (2020) 3

    Khosla,P.,Teterwak,P.,Wang,C.,Sarna,A.,Tian,Y.,Isola,P.,Maschinot, A., Liu, C., Krishnan, D.: Supervised contrastive learning. NeurIPS 33, 18661–18673 (2020) 3

    2020

  14. [22]

    arXiv preprint arXiv:2401.09794 (2024) 5

    Koo, G., Yoon, S., Yoo, C.D.: Wavelet-guided acceleration of text inversion in diffusion-based image editing. arXiv preprint arXiv:2401.09794 (2024) 5

    2024 arXiv

  15. [23]

    In: IEEE Workshop on 3D Representation and Recognition (3dRR-13) (2013) 10, 23

    Krause, J., Stark, M., Deng, J., Fei-Fei, L.: 3d object representations for fine-grained categorization. In: IEEE Workshop on 3D Representation and Recognition (3dRR-13) (2013) 10, 23

    2013

  16. [24]

    NeurIPS32 (2019) 4

    Kull, M., Perello Nieto, M., Kängsepp, M., Silva Filho, T., Song, H., Flach, P.: Beyond temperature scaling: Obtaining well-calibrated multi-class prob- abilities with dirichlet calibration. NeurIPS32 (2019) 4

    2019

  17. [25]

    In: EMNLP

    Lester, B., Al-Rfou, R., Constant, N.: The power of scale for parameter- efficient prompt tuning. In: EMNLP. pp. 3045–3059 (Nov 2021) 5

    2021

  18. [26]

    arXiv preprint arXiv:2303.12748 (2023) 14

    LeVine, W., Pikus, B., Raja, P., Gil, F.A.: Enabling calibration in the zero-shot inference of large vision-language models. arXiv preprint arXiv:2303.12748 (2023) 14

    2023 arXiv

  19. [27]

    Li, F.F., Andreeto, M., Ranzato, M., Perona, P.: Caltech 101 (Apr 2022) 23

    2022

  20. [28]

    ACM Computing Surveys55(9), 1–35 (2023) 4

    Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., Neubig, G.: Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Computing Surveys55(9), 1–35 (2023) 4

    2023

  21. [29]

    Liu, X., Ji, K., Fu, Y., Tam, W.L., Du, Z., Yang, Z., Tang, J.: P-tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks (2022) 4

    2022

  22. [30]

    Journal of Machine Learning Research15(93), 3221–3245 (2014) 23

    van der Maaten, L.: Accelerating t-sne using tree-based algorithms. Journal of Machine Learning Research15(93), 3221–3245 (2014) 23

    2014

  23. [31]

    Maji, S., Kannala, J., Rahtu, E., Blaschko, M., Vedaldi, A.: Fine-grained visual classification of aircraft. Tech. rep. (2013) 10, 23 Prompting without Panic: Attribute-aware, Zero-shot, Test-Time Calibration 17

    2013

  24. [32]

    In: NeurIPS (2022) 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 19, 22

    Manli, S., Weili, N., De-An, H., Zhiding, Y., Tom, G., Anima, A., Chaowei, X.: Test-time prompt tuning for zero-shot generalization in vision-language models. In: NeurIPS (2022) 1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 19, 22

    2022

  25. [33]

    In: AAAI

    Naeini,M.P.,Cooper,G.F.,Hauskrecht,M.:Obtainingwellcalibratedprob- abilities using bayesian binning. In: AAAI. p. 2901–2907 (2015) 5

    2015

  26. [34]

    In: ICVGIP

    Nilsback, M., Zisserman, A.: Automated flower classification over a large number of classes. In: ICVGIP. pp. 722–729 (2008) 6, 10, 23, 24

    2008

  27. [35]

    In: ICLR

    Niu, S., Wu, J., Zhang, Y., Chen, Y., Zheng, S., Zhao, P., Tan, M.: Effi- cient test-time model adaptation without forgetting. In: ICLR. vol. 162, pp. 16888–16905 (17–23 Jul 2022) 2

    2022

  28. [36]

    Advances in Neural Information Processing Systems37, 12677–12707 (2024) 14

    Oh, C., Lim, H., Kim, M., Han, D., Yun, S., Choo, J., Hauptmann, A., Cheng, Z.Q., Song, K.: Towards calibrated robust fine-tuning of vision- language models. Advances in Neural Information Processing Systems37, 12677–12707 (2024) 14

    2024

  29. [37]

    In: ICCV

    Park,H.,Noh,J.,Oh,Y.,Baek,D.,Ham,B.:Acls:Adaptiveandconditional label smoothing for network calibration. In: ICCV. pp. 3936–3945 (2023) 4

    2023

  30. [38]

    In: CVPR (2012) 10, 23, 24

    Parkhi, O., Vedaldi, A., A.Zisserman, Jawahar, C.V.: Cats and dogs. In: CVPR (2012) 10, 23, 24

    2012

  31. [39]

    In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)

    Patra, R., Hebbalaguppe, R., Dash, T., Shroff, G., Vig, L.: Calibrating deep neural networks using explicit regularisation and dynamic data pruning. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). pp. 1541–1549 (January 2023) 4

    2023

  32. [40]

    arXiv preprint arXiv:1701.06548 (2017) 4

    Pereyra, G., Tucker, G., Chorowski, J., Kaiser, Ł., Hinton, G.: Regularizing neural networks by penalizing confident output distributions. arXiv preprint arXiv:1701.06548 (2017) 4

    2017 arXiv

  33. [41]

    In: ADVANCES IN LARGE MARGIN CLASSIFIERS

    Platt, J.C.: Probabilistic outputs for support vector machines and com- parisons to regularized likelihood methods. In: ADVANCES IN LARGE MARGIN CLASSIFIERS. pp. 61–74. MIT Press (1999) 4

    1999

  34. [42]

    In: ICCV

    Pratt, S., Covert, I., Liu, R., Farhadi, A.: What does a platypus look like? generating customized prompts for zero-shot image classification. In: ICCV. pp. 15691–15701 (2023) 6

    2023

  35. [43]

    In: ICML

    Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al.: Learning transferable visual models from natural language supervision. In: ICML. pp. 8748–8763 (2021) 5, 11, 22

    2021

  36. [44]

    arXiv preprint arXiv:2111.00506 (2021) 4

    Rawat, M., Hebbalaguppe, R., Vig, L.: Pnpood: Out-of-distribution de- tection for text classification via plug andplay data augmentation. arXiv preprint arXiv:2111.00506 (2021) 4

    2021 arXiv

  37. [45]

    Recht, B., Roelofs, R., Schmidt, L., Shankar, V.: Do imagenet classifiers generalize to imagenet? In: ICML. pp. 5389–5400 (2019) 10, 22, 23

    2019

  38. [46]

    In: EMNLP (2024) 2

    S., G., Basu, S., Feizi, S., Manocha, D.: Intcoop: Interpretability-aware vision-language prompt tuning. In: EMNLP (2024) 2

    2024

  39. [47]

    NeurIPS36 (2024) 1, 2, 3, 4, 10, 14, 19, 20, 22, 24 18 Hebbalaguppe et al

    Samadh, A.: Align your prompts: Test-time prompting with distribution alignment for zero-shot generalization. NeurIPS36 (2024) 1, 2, 3, 4, 10, 14, 19, 20, 22, 24 18 Hebbalaguppe et al

    2024

  40. [48]

    CoRRabs/1212.0402 (2012) 10, 23, 24

    Soomro, K., Zamir, A.R., Shah, M.: UCF101: A dataset of 101 human ac- tions classes from videos in the wild. CoRRabs/1212.0402 (2012) 10, 23, 24

    2012 arXiv

  41. [49]

    In: CVPR

    Tian, X., Zou, S., Yang, Z., Zhang, J.: Argue: Attribute-guided prompt tuning for vision-language models. In: CVPR. pp. 28578–28587 (2024) 2, 6, 20

    2024

  42. [50]

    Tu, W., Deng, W., Campbell, D., Gould, S., Gedeon, T.: An empirical study into what matters for calibrating vision-language models (2024) 1

    2024

  43. [51]

    NeurIPS34, 11809–11820 (2021) 4

    Wang, D.B., Feng, L., Zhang, M.L.: Rethinking calibration of deep neural networks: Do not be afraid of overconfidence. NeurIPS34, 11809–11820 (2021) 4

    2021

  44. [52]

    In: NeurIPS (2019) 10, 22, 23

    Wang, H., Ge, S., Lipton, Z., Xing, E.P.: Learning robust global represen- tations by penalizing local predictive power. In: NeurIPS (2019) 10, 22, 23

    2019

  45. [53]

    arXiv preprint arXiv:2402.04655 (2024) 14

    Wang, S., Wang, J., Wang, G., Zhang, B., Zhou, K., Wei, H.: Open- vocabulary calibration for fine-tuned clip. arXiv preprint arXiv:2402.04655 (2024) 14

    2024 arXiv

  46. [54]

    In: CVPR

    Xiao, J., Hays, J., Ehinger, K.A., Oliva, A., Torralba, A.: Sun database: Large-scale scene recognition from abbey to zoo. In: CVPR. pp. 3485–3492 (2010) 10, 23

    2010

  47. [55]

    In: CHI Conference on Hu- man Factors in Computing Systems (2024) 1

    Y., N.t.: Multimodal healthcare ai: Identifying and designing clinically rele- vant vision-language applications for radiology. In: CHI Conference on Hu- man Factors in Computing Systems (2024) 1

    2024

  48. [56]

    arXiv preprint arXiv:2402.01735 (2024) 1

    Yi, Z., Yilin, Z., Rong, X., Jing, L., Hillming, L.: Vialm: A survey and benchmark of visually impaired assistance with large models. arXiv preprint arXiv:2402.01735 (2024) 1

    2024 arXiv

  49. [57]

    Yoon, E., Yoon, H.S., Harvill, J., Hasegawa-Johnson, M., Yoo, C.D.: IN- Tapt: Information-theoretic adversarial prompt tuning for enhanced non- native speech recognition (2023) 5

    2023

  50. [58]

    In: ICLR (2024) 1, 2, 3, 4, 8, 9, 11, 13, 14, 19, 20, 21, 22, 24

    Yoon, H.S., Yoon, E., Tee, J.T.J., Hasegawa-Johnson, M.A., Li, Y., Yoo, C.D.: C-TPT: Calibrated test-time prompt tuning for vision-language mod- els via text feature dispersion. In: ICLR (2024) 1, 2, 3, 4, 8, 9, 11, 13, 14, 19, 20, 21, 22, 24

    2024

  51. [59]

    arXiv preprint arXiv:2312.06708 (2023) 5

    Yoon, S., Koo, G., Hong, J.W., Yoo, C.D.: Neutral editing framework for diffusion-based video editing. arXiv preprint arXiv:2312.06708 (2023) 5

    2023 arXiv

  52. [60]

    arXiv preprint arXiv:2410.15430 (2024) 3

    Zhang, T., Wang, J., Guo, H., Dai, T., Chen, B., Xia, S.T.: Boostadapter: Improving vision-language test-time adaptation via regional bootstrapping. arXiv preprint arXiv:2410.15430 (2024) 3

    2024 arXiv

  53. [61]

    In: CVPR (2022) 4, 5, 9

    Zhou, K., Yang, J., Loy, C.C., Liu, Z.: Conditional prompt learning for vision-language models. In: CVPR (2022) 4, 5, 9

    2022

  54. [62]

    Zhou, K., Yang, J., Loy, C.C., Liu, Z.: Learning to prompt for vision- language models. IJCV (2022) 4, 5, 9 Prompting without Panic: Attribute-aware, Zero-shot, Test-Time Calibration 19 6 Supplemental material To keep the main manuscript self-contained, we include the followin...

    2022

  55. [63]

    natural adversarial examples

    across 10 datasets, and we compare its performance with our pro- posed approach, TCA. Our findings demonstrate that integratingTCA with PromptAlign[47] leads to a reduction in calibration error and an improve- ment in accuracy. Additionally, we provide t-SNE visualizations to ...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.