Pith. sign in

REVIEW 4 major objections 5 minor 66 references

BayesAdapter: enhanced uncertainty estimation in CLIP few-shot adaptation

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A Bayesian posterior over adapter weights produces better-calibrated CLIP few-shot predictions.

desk verdict A useful recasting of CLAP as MAP inference with a sensible variational extension, but the abstract's coverage claim is internally inconsistent with the paper's own ablation table. read the letter →

arxiv 2412.09718 v2 pith:DWACJVQ2 submitted 2024-12-12 cs.CV cs.LG

Pablo Morales-Álvarez , Stergios Christodoulidis , Maria Vakalopoulou , Pablo Piantanida , Jose Dolz This is my paper · ORCID
classification cs.CVcs.LG
keywords CLIPfew-shotadaptationvision-languagemodelsBayesiandeeplearningvariationalinferencecalibrationselectiveclassificationuncertaintyestimation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that the best-accuracy CLIP adapters are not the best at knowing when they are right, and that this gap can be closed by treating the adapter weights as a probability distribution instead of a single fitted matrix. Its starting observation is that CLAP, a state-of-the-art few-shot adapter, is exactly maximum-a-posteriori inference in a probabilistic model whose Gaussian prior is centered on the zero-shot text prototypes and whose likelihood is a multinomial softmax. BayesAdapter replaces that point estimate with a variational Gaussian posterior over the weights and averages predictions over it. Across 11 datasets, six shot levels, and two visual backbones, the paper reports consistently better calibration (expected calibration error 4.151 versus 6.592 on ResNet-50) and higher reliable test coverage (13.423% versus 7.103% at 99% confidence), while staying within about 0.7% in accuracy; at 32 shots the accuracy gap disappears and BayesAdapter leads on all metrics.

What carries the argument

The object that carries the argument is the variational Gaussian posterior $q(W)=\mathcal{N}(W|\Omega,\Sigma)$ over the adapter weight matrix $W$, a block-diagonal covariance matrix with a separate learned variance for each class. Its prior $p(W)=\mathcal{N}(W|T,\Lambda)$, centered on the zero-shot class prototypes $T$, is chosen so that MAP inference in the same probabilistic model reproduces CLAP exactly (Proposition 1); the variational step then replaces the MAP mode with a distribution whose per-class spread is learned from the few-shot data. Training minimizes the negative evidence lower bound, whose data term is the expectation of the cross-entropy under $q$ estimated with $s_{\mathrm{MC}}=3$ Monte Carlo samples via the reparameterization trick, and whose regularizer is the closed-form KL divergence between two Gaussians — the probabilistic analogue of CLAP's cross-entropy plus $\ell^2$ penalty. At test time, softmax probabilities are averaged over the posterior by Monte Carlo integration, which is the step that converts the learned parameter distribution into calibrated confidence scores.

What would settle it

Run BayesAdapter on a held-out dataset whose zero-shot accuracy differs clearly from the average of the 11 datasets used to set the prior, keep the prior standard deviation fixed at 0.01 without consulting that dataset's test curves, and measure ECE and coverage at 99% confidence against CLAP. The ablation in Table 4 shows the prior governs the reported gains (coverage moves from 3.1% to 12.0% as the standard deviation goes from 0.0001 to 1.0), so a dataset where the fixed prior is miscalibrated relative to its zero-shot level should shrink the ~2.4-point ECE gap or erase the coverage advantage; finding no such dataset across several held-out tasks would instead support the validation-free claim.

Watch

Extended reading notes

Core claim

The central claim is that uncertainty in CLIP few-shot adaptation improves when the adapter is fitted by variational Bayes rather than by MAP inference. Proposition 1 shows that the CLAP objective — cross-entropy plus per-class $\ell^2$ penalties pulling the linear layer $W$ toward the zero-shot prototypes $T$ — is exactly the negative log posterior of the model $p(W)=\mathcal{N}(W|T,\Lambda)$ with a multinomial softmax likelihood, so CLAP keeps only the mode of the posterior and discards all other information about parameter uncertainty. BayesAdapter instead fits $q(W)=\mathcal{N}(W|\Omega,\Sigma)$, a Gaussian posterior with a block-diagonal covariance carrying one variance per class, by minimizing the negative evidence lower bound using the reparameterization trick, and integrates over the posterior at test time via Monte Carlo sampling. The empirical claim is that this parameter-level uncertainty propagates into the predictions: BayesAdapter has the best expected and adaptive calibration error on both ResNet-50 and ViT-B/16, is reliable at every confidence level from 99% down to 80% while covering more test samples and more classes than all seven baselines, and pays roughly a 0.5–0.7% accuracy penalty in the low-shot regime that becomes an accuracy advantage by 32 shots.

Load-bearing premise

The load-bearing premise is that a prior standard deviation of 0.01, fixed once from the average zero-shot accuracy over the same 11 benchmark datasets, transfers to new tasks without per-task validation; the paper's own ablation shows that this single hyperparameter swings accuracy between 58.7 and 69.4 and swings 99%-confidence coverage between 3.1% and 12.0%.

Editorial extensions

If this is right

  • Averaged over the 11 datasets and all shot levels, BayesAdapter is the only compared method that is reliable at every confidence level from 99% down to 80%, and at 99% confidence it covers 13.4% of the ResNet-50 test set versus 7.1% for CLAP, letting a safety-critical system act automatically on roughly twice as many predictions.
  • The accuracy gap shrinks with more data: the paper reports that with 32 shots BayesAdapter is the best method in accuracy as well as in calibration and coverage, so the probabilistic formulation does not trade away discriminative power once the posterior variance can be estimated.
  • High-confidence selections are more class-diverse under BayesAdapter — 61.1% of classes represented versus 38.5% for CLAP at 99% confidence on ResNet-50 — so the abstention behavior does not quietly concentrate on particular categories.
  • Because $s_{\mathrm{MC}}=3$ Monte Carlo samples (or even 1) suffice during training, the better uncertainty costs little: on Caltech101 the training time is 42.4 seconds versus 32.3 seconds for CLAP at 32 shots, a cheaper route to calibrated confidence than ensembling.
  • The paper suggests that the number of available shots can serve as the practical switch: keep the deterministic CLAP when data are extremely scarce (around 1 shot per class) and use BayesAdapter when there are enough shots to learn the posterior variance.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The prior variance is a single global knob chosen using the evaluation benchmarks themselves; on a genuinely new dataset the fixed value 0.01 could be off, and the learned per-class variances suggest a natural repair — estimating the prior from the support set's zero-shot confidence — that the paper does not test.
  • The per-class variances $\sigma_1^2,\dots,\sigma_C^2$ learned by the posterior are themselves a reliability signal for each category, which could be reused for out-of-distribution detection or for choosing which images to label next in an active-learning loop; the paper only uses them to shape the predictive distribution.
  • Any adapter whose loss is cross-entropy plus an $\ell^2$ pull toward a reference point admits the same variational treatment, so the argument should extend to adapters beyond CLAP even though the paper demonstrates it only for that one.
  • The EuroSAT 4-shot example — CLAP and LP++ abstain completely at 99% confidence while BayesAdapter covers 10.5% — suggests the benefit is largest precisely where deterministic adapters become over-conservative, so datasets with low-confidence support sets are where the method is most likely to matter in deployment.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper studies uncertainty estimation in few-shot adaptation of CLIP. It observes that CLAP, a state-of-the-art deterministic adapter, can be viewed as MAP inference in a probabilistic model with a Gaussian prior and a multinomial likelihood, and proposes BayesAdapter, a variational Bayes extension that learns a Gaussian posterior over the adapter weights instead of a point estimate. The empirical evaluation covers 11 datasets, two backbones, six shot levels, and three seeds, and reports that BayesAdapter improves calibration and selective-classification coverage over CLAP and six other adapters while remaining competitive in accuracy. The supplementary material includes full per-dataset tables with standard errors, ablations, and training-time comparisons.

Significance. BayesAdapter is conceptually simple and plausible: replacing a point estimate with a variational posterior for a linear adapter is a natural step, and Proposition 1, once the covariance convention in Eq. (3) is corrected, gives a clean probabilistic interpretation of CLAP. The paper's strongest assets are the breadth of the evaluation, the availability of detailed per-dataset tables, and the inclusion of standard errors and ablations. The calibration gains (ECE 4.151 vs 6.592 for CLAP on ResNet-50) are supported by the reported numbers. However, the headline selective-coverage advantage is internally inconsistent across Table 2, Table 4, and Appendix Table 11, and the claimed validation-free protocol is compromised by choosing the prior standard deviation based on the evaluation datasets. These issues are load-bearing for the paper's central claims and must be resolved before the empirical conclusions can be taken at face value.

major comments (4)
  1. [§4.2.2, Tables 2 and 4] Table 2 reports 13.423% test-set coverage at 99% confidence for BayesAdapter on ResNet-50, but Table 4 reports Cov@99%=9.213 for the same configuration (prior std=0.01, sMC=3), with both tables described as aggregated over the same datasets, shot levels, and seeds. Recomputing from Appendix Table 11 with ✗ treated as zero gives approximately 11.6%, which matches neither value. The corresponding CLAP entries are also inconsistent: Table 11 averages to about 6.6%, not the 7.103% reported in Table 2. Since the 6.3-point coverage gain over CLAP is a central contribution, the averaging rule must be clarified and the tables corrected; if the correct value is around 9.2%, the advantage over CLAP shrinks to roughly 2.1 points, materially weakening the claim.
  2. [§4.2.2 and Appendix Table 11] The text states that BayesAdapter 'is reliable at all confidence levels,' but Table 11 contains many ✗ entries for BayesAdapter, for example EuroSAT, ImageNet, and SUN397 at 1 shot. Either the statement refers to an aggregate notion of reliability that is not defined, or it is contradicted by the per-configuration table. This ambiguity also affects how the numbers in Table 2 should be interpreted, and a precise definition of how ✗ entries enter any average is needed.
  3. [§4.1 and §4.2.4, Table 4] The prior standard deviation is chosen 'based on the average empirical performance of the ZS classification' on the same 11 benchmark datasets and then fixed. This constitutes model selection on the evaluation data and is at odds with the stated validation-free protocol in Section 4.1, which says no validation or test samples are accessible. Since Table 4 shows that coverage at 99% confidence ranges from 3.062% to 12.043% across prior standard deviations, the reported gains are not robustly decoupled from this hyperparameter choice. I ask the authors to select σ_p without reference to test performance, for example from the few-shot support set or from a fixed a-priori value, or to explicitly acknowledge and quantify the resulting optimism.
  4. [Eq. (3) and Proposition 1] The prior covariance written in Eq. (3) is Λ=2·diag(λ_1^{-1},...,λ_C^{-1}) with each entry repeated D times. With this convention, log p(W) equals -Σ_c (λ_c/4)||w_c-t_c||^2, not the CLAP regularizer -Σ_c λ_c||w_c-t_c||^2. The proof in Appendix A.1 uses Λ=0.5·diag(...), which is the correct expression for the stated equivalence. Eq. (3) should be corrected to match the appendix, or the notation should be changed to an inverse-covariance parameterization; otherwise Proposition 1 does not follow from the model as defined.
minor comments (5)
  1. [Throughout] There are several typos, including 'apdaters' in Section 4.1, 'probablity' in Proposition 1, and 'Worskshops' in reference [11].
  2. [Figure 3 and Table 11] Figure 3 states that CLAP obtains 0.00% coverage on EuroSAT with 4 shots, while Table 11 reports 0.03±0.02 for CLAP at that configuration; please clarify whether the figure is from a single seed and state which seed.
  3. [Table 4 caption] The caption says results are 'similar to Table 1', but the numbers for the common configuration differ slightly from Table 1, for example accuracy 69.437 vs 69.476; please specify whether Table 4 comes from separate runs and report standard errors.
  4. [§4.1] The linear KL annealing schedule is mentioned but its setting is not detailed, and its sensitivity is not studied; given the paper's emphasis on validation-free adaptation, a sentence on how the schedule was chosen would help.
  5. [Abstract] The abstract's '∼6-9% gain in test set coverage at 99% confidence' is not directly traceable to a single table because the ResNet-50 and ViT-16 values differ; consider reporting the range explicitly in the abstract.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: BayesAdapter's variational derivation is self-contained, and the reported uncertainty gains are empirical rather than forced by construction.

full rationale

The core derivation is self-contained. Proposition 1 is proved in Appendix A.1 by expanding the Gaussian prior and multinomial likelihood, and the ELBO objective in Eq. (5) follows from the standard KL/ELBO identity in Appendix A.2; neither step imports an unverified result or a self-citation. The variational posterior, reparameterized Monte Carlo sampling, and MC predictive integration are standard and do not reduce to the reported ECE or coverage values. The comparison against CLAP is fair and externally grounded: the CLAP objective is quoted in Eq. (1), and the paper's own derivation shows it equals MAP in their model, so the MAP-to-Bayes extension does not depend on a self-citation chain. The prior standard deviation is set to 0.01 'based on the average empirical performance of the ZS classification' on the same benchmark datasets; this is a hyperparameter-selection concern, and the discrepancy between Table 2 (Cov@99% = 13.423) and Table 4 (Cov@99% = 9.213 for the same configuration) is an internal-consistency issue, but neither constitutes circularity because the reported calibration/coverage metrics are not mathematically defined by, nor forced to equal, that scalar choice or the zero-shot accuracy. No fitted parameter is renamed as a prediction, no uniqueness theorem is imported from the authors' prior work, and no ansatz is smuggled in via citation. The stated 1-shot limitation is a performance caveat rather than a circular step. Therefore the derivation chain is not circular.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on a standard probabilistic model and variational inference machinery. No new entities are introduced. The most significant free parameter is the prior variance, which is tuned on the evaluation benchmarks, and the main assumptions concern the adequacy of the variational approximation and MC sampling.

free parameters (3)
  • prior standard deviation (σ_p) = 0.01
    Chosen based on average empirical ZS performance on the 11 benchmark datasets (Sec. 4.1), not a held-out split. Table 4 shows it strongly affects accuracy and coverage.
  • number of MC samples (sMC) = 3
    Ablated in Table 4; results are robust across 1-10, but it remains a hyperparameter of the estimator.
  • KL annealing schedule = linear (not fully specified)
    Follows common practice [21], but the exact schedule and its interaction with the prior are not detailed.
assumptions (4)
  • domain assumption The probabilistic model in Eqs. (3)-(4), with a Gaussian prior over W and a multinomial likelihood, is an appropriate generative model for the few-shot classification task.
    The paper posits this model and shows CLAP is its MAP solution, but the model itself is a modeling choice, not a consequence.
  • domain assumption The variational posterior with class-wise diagonal Gaussian covariance is a sufficiently accurate approximation to the true posterior for the purpose of calibration and selective classification.
    The true posterior is intractable; the diagonal restriction limits the approximation, and the paper does not quantify the approximation error.
  • standard math Monte Carlo sampling with 3 samples during training and at inference produces low-variance estimates of the ELBO and the predictive distribution.
    Standard in variational inference; the paper ablated sMC but did not analyze estimation variance.
  • standard math The CLIP visual and text encoders are frozen, and their embeddings are treated as deterministic inputs to the adapter.
    This is the standard adapter setting and is stated in Section 3.1.

how reviews work

0 comments
Cite this review

Pith. "Pith review of BayesAdapter: enhanced uncertainty estimation in CLIP few-shot adaptation." pith.science (2026). https://pith.science/paper/DWACJVQ2

@misc{pith2026241209718,
  author       = {Pith},
  title        = {Pith review of: BayesAdapter: enhanced uncertainty estimation in CLIP few-shot adaptation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DWACJVQ2}},
  note         = {Machine review of arXiv:2412.09718}
}
read the original abstract

The emergence of large pre-trained vision-language models (VLMs) represents a paradigm shift in machine learning, with unprecedented results in a broad span of visual recognition tasks. CLIP, one of the most popular VLMs, has exhibited remarkable zero-shot and transfer learning capabilities in classification. To transfer CLIP to downstream tasks, adapters constitute a parameter-efficient approach that avoids backpropagation through the large model (unlike related prompt learning methods). However, CLIP adapters have been developed to target discriminative performance, and the quality of their uncertainty estimates has been overlooked. In this work we show that the discriminative performance of state-of-the-art CLIP adapters does not always correlate with their uncertainty estimation capabilities, which are essential for a safe deployment in real-world scenarios. We also demonstrate that one of such adapters is obtained through MAP inference from a more general probabilistic framework. Based on this observation we introduce BayesAdapter, which leverages Bayesian inference to estimate a full probability distribution instead of a single point, better capturing the variability inherent in the parameter space. In a comprehensive empirical evaluation we show that our approach obtains high quality uncertainty estimates in the predictions, standing out in calibration and selective classification. Our code will be publicly available upon acceptance of the paper.

Figures

Figures reproduced from arXiv: 2412.09718 by the authors.

Figure 1
Figure 1. Graphical representation of the novel BayesAdapter. [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Calibration plots for the four best methods in terms of ECE in [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Visualizing the over-conservative behavior of some baselines, including the most recent ones CLAP and LP++. We show the histogram of the confidence score on the test samples (after adapting in a few-shot training data). Whereas CLAP, LP++ and TipA abstain from making predictions at 99% confidence, BayesAdapter does cover 10.53% of the test set, achieving accuracy above 99%. TipA-f-, CrossModal and TaskRes also make … view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Evolution of metrics with shots. From left to right: accuracy, calibration, and coverage at 99% confidence level. In the last one, the marker is missing if the method is not reliable at 99% confidence level. Whereas methods are similar in accuracy, more differences app…
Figure 5
Figure 5. Figure 5: Calibration plots for all the compared methods (ResNet50 backbone). In each case, the lower subplot depicts the accuracy and average confidence for samples in each one of the ten bins (from 0% to 100% of confidence score by steps of 10%). Ideally, the gap between them …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

66 extracted references · 59 canonical work pages

  1. [1]

    Vari- ational inference: A review for statisticians

    David M Blei, Alp Kucukelbir, and Jon D McAuliffe. Vari- ational inference: A review for statisticians. Journal of the American statistical Association , 112(518):859–877, 2017. 2, 4

    work page 2017

  2. [2]

    Food-101 – mining discriminative components with ran- dom forests

    Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. Food-101 – mining discriminative components with ran- dom forests. In European Conference on Computer Vision (ECCV), 2014. 5

    work page 2014

  3. [3]

    Emerg- ing properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv ´e J´egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerg- ing properties in self-supervised vision transformers. In Pro- ceedings of the IEEE/CVF international conference on com- puter vision, pages 9650–9660, 2021. 3

    2021

  4. [4]

    Castro-Mac ´ıas, Pablo Morales- ´Alvarez, Yunan Wu, Rafael Molina, and Aggelos K

    Francisco M. Castro-Mac ´ıas, Pablo Morales- ´Alvarez, Yunan Wu, Rafael Molina, and Aggelos K. Katsaggelos. Hyper- bolic secant representation of the logistic function: Appli- cation to probabilistic multiple instance learning for ct in- tracranial hemorrhage detection. Artificial Intelligence, 331: 104115, 2024. 4

    work page 2024

  5. [5]

    An empirical study of training self-supervised vision transformers

    Xinlei Chen, Saining Xie, and Kaiming He. An empirical study of training self-supervised vision transformers. InPro- ceedings of the IEEE/CVF international conference on com- puter vision, pages 9640–9649, 2021. 3

    2021

  6. [6]

    Describing textures in the wild

    Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. Describing textures in the wild. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR), pages 3606– 3613, 2014. 5

    work page 2014

  7. [7]

    Imagenet: A large-scale hierarchical im- age database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical im- age database. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 248–255, 2009. 5

    work page 2009

  8. [8]

    Turrisi da Costa, Cees G.M

    Mohammad Mahdi Derakhshani, Enrique Sanchez, Adrian Bulat, Victor G. Turrisi da Costa, Cees G.M. Snoek, Geor- gios Tzimiropoulos, and Brais Martinez. Bayesian prompt learning for image-language model generalization. In Pro- ceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 15237–15246, 2023. 3

    work page 2023

Show all 66 references
  1. [9]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition ...

    2021

  2. [10]

    Improving clip training with language rewrites

    Lijie Fan, Dilip Krishnan, Phillip Isola, Dina Katabi, and Yonglong Tian. Improving clip training with language rewrites. Advances in Neural Information Processing Sys- tems, 36, 2024. 3

    2024

  3. [11]

    Fergus, and P

    Li Fei-Fei, R. Fergus, and P. Perona. Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Worskshops (CVPRW), page...

    2004

  4. [12]

    Clipscope: Enhancing zero-shot ood detection with bayesian scoring

    Hao Fu, Naman Patel, Prashanth Krishnamurthy, and Farshad Khorrami. Clipscope: Enhancing zero-shot ood detection with bayesian scoring. arXiv preprint arXiv:2405.14737, 2024. 3

    2024 arXiv

  5. [13]

    Clip-adapter: Better vision-language models with feature adapters

    Peng Gao, Shijie Geng, Renrui Zhang, Teli Ma, Rongyao Fang, Yongfeng Zhang, Hongsheng Li, and Yu Qiao. Clip-adapter: Better vision-language models with feature adapters. International Journal of Computer Vision (IJCV),

  6. [14]

    Selective classification for deep neural networks

    Yonatan Geifman and Ran El-Yaniv. Selective classification for deep neural networks. InAdvances in Neural Information Processing Systems. Curran Associates, Inc., 2017. 6

    2017

  7. [15]

    A data-driven measure of relative uncertainty for misclassification detection

    Eduardo Dadalto C ˆamara Gomes, Marco Romanelli, Georg Pichler, and Pablo Piantanida. A data-driven measure of relative uncertainty for misclassification detection. In The 9 Twelfth International Conference on Learning Representa- tions, 2024. 2, 6

    2024

  8. [16]

    Weinberger

    Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. On calibration of modern neural networks. In Proceedings of the 34th International Conference on Machine Learning , pages 1321–1330. PMLR, 2017. 1, 6

    2017

  9. [17]

    Pay attention to your neighbours: Training-free open-vocabulary semantic segmentation

    Sina Hajimiri, Ismail Ben Ayed, and Jose Dolz. Pay attention to your neighbours: Training-free open-vocabulary semantic segmentation. In IEEE/CVF Winter Conference on Applica- tions of Computer Vision (WACV), 2024. 1

    2024

  10. [18]

    Visual-language prompt tuning with knowledge-guided context optimization

    Changsheng Xu Hantao Yao, Rui Zhang. Visual-language prompt tuning with knowledge-guided context optimization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023. 3

    2023

  11. [19]

    Deep residual learning for image recognition.Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2016

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition.Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2016. 5

    2016

  12. [20]

    Introducing eurosat: A novel dataset and deep learning benchmark for land use and land cover clas- sification

    Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth. Introducing eurosat: A novel dataset and deep learning benchmark for land use and land cover clas- sification. In IEEE International Geoscience and Remote Sensing Symposium (IGARSS), pages 3606–3613, 2018. 5

    2018

  13. [21]

    Improving explorability in variational inference with annealed variational objectives

    Chin-Wei Huang, Shawn Tan, Alexandre Lacoste, and Aaron C Courville. Improving explorability in variational inference with annealed variational objectives. Advances in neural information processing systems, 31, 2018. 6

    2018

  14. [22]

    Lp++: A surprisingly strong linear probe for few-shot clip

    Yunshi Huang, Fereshteh Shakeri, Jose Dolz, Malik Boudiaf, Houda Bahig, and Ismail Ben Ayed. Lp++: A surprisingly strong linear probe for few-shot clip. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 23773–23782, 2024. 1, 2, 5

    2024

  15. [23]

    What are bayesian neural network posteriors really like? In International conference on machine learning, pages 4629–4640

    Pavel Izmailov, Sharad Vikram, Matthew D Hoffman, and Andrew Gordon Gordon Wilson. What are bayesian neural network posteriors really like? In International conference on machine learning, pages 4629–4640. PMLR, 2021. 3

    2021

  16. [24]

    On uncertainty, tempering, and data aug- mentation in bayesian classification

    Sanyam Kapoor, Wesley J Maddox, Pavel Izmailov, and An- drew G Wilson. On uncertainty, tempering, and data aug- mentation in bayesian classification. Advances in Neural In- formation Processing Systems, 35:18211–18225, 2022. 3

    2022

  17. [25]

    Consistency and uncertainty: Iden- tifying unreliable responses from black-box vision-language models for selective visual question answering

    Zaid Khan and Yun Fu. Consistency and uncertainty: Iden- tifying unreliable responses from black-box vision-language models for selective visual question answering. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10854–10863, 2024. 2, 3

    2024

  18. [26]

    Maple: Multi-modal prompt learning

    Muhammad Uzair Khattak, Hanoona Rasheed, Muhammad Maaz, Salman Khan, and Fahad Shahbaz Khan. Maple: Multi-modal prompt learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 19113–19122, 2023. 2

    2023

  19. [27]

    Variational diffusion models

    Diederik Kingma, Tim Salimans, Ben Poole, and Jonathan Ho. Variational diffusion models. Advances in neural infor- mation processing systems, 34:21696–21707, 2021. 4

    2021

  20. [28]

    Auto-encoding variational bayes

    Diederik P Kingma. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013. 4

    2013 arXiv

  21. [29]

    3d object representations for fine-grained categorization

    Jonathan Krause, Michael Stark, Jia Deng, and Li Fei- Fei. 3d object representations for fine-grained categorization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , page 3498–3505,

  22. [30]

    Effective adaptation in multi- task co-training for unified autonomous driving

    Xiwen Liang, Yangxin Wu, Jianhua Han, Hang Xu, Chun- jing Xu, and Xiaodan Liang. Effective adaptation in multi- task co-training for unified autonomous driving. Advances in Neural Information Processing Systems (NeurIPS) , 35: 19645–19658, 2022. 1

    2022

  23. [31]

    Multimodality helps unimodality: Cross- modal few-shot learning with multimodal models

    Zhiqiu Lin, Samuel Yu, Zhiyi Kuang, Deepak Pathak, and Deva Ramanan. Multimodality helps unimodality: Cross- modal few-shot learning with multimodal models. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023. 1, 2, 5

    2023

  24. [32]

    Clip-driven universal model for organ segmentation and tumor detection

    Jie Liu, Yixiao Zhang, Jie-Neng Chen, Junfei Xiao, Yongyi Lu, Bennett A Landman, Yixuan Yuan, Alan Yuille, Yucheng Tang, and Zongwei Zhou. Clip-driven universal model for organ segmentation and tumor detection. In Proceedings of the IEEE/CVF International Conference on Compute...

    2023

  25. [33]

    Energy-based out-of-distribution detection

    Weitang Liu, Xiaoyun Wang, John Owens, and Yixuan Li. Energy-based out-of-distribution detection. In Advances in Neural Information Processing Systems , pages 21464– 21475. Curran Associates, Inc., 2020. 2

    2020

  26. [34]

    S. Maji, J. Kannala, E. Rahtu, M. Blaschko, and A. Vedaldi. Fine-grained visual classification of aircraft. In ArXiv Preprint, 2013. 5

    2013

  27. [35]

    Bayesian exploration of pre-trained models for low-shot image classi- fication

    Yibo Miao, Yu Lei, Feng Zhou, and Zhijie Deng. Bayesian exploration of pre-trained models for low-shot image classi- fication. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 23849– 23859, 2024. 3

    2024

  28. [36]

    Activation- level uncertainty in deep neural networks

    Pablo Morales-Alvarez, Daniel Hern ´andez-Lobato, Rafael Molina, and Jos ´e Miguel Hern ´andez-Lobato. Activation- level uncertainty in deep neural networks. In International Conference on Learning Representations, 2020. 2

    2020

  29. [37]

    Katsaggelos

    Pablo Morales- ´Alvarez, Pablo Ruiz, Scott Coughlin, Rafael Molina, and Aggelos K. Katsaggelos. Scalable variational gaussian processes for crowdsourcing: Glitch detection in ligo. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(3):1534–1551, 2022. 4

    2022

  30. [38]

    Robust calibration of large vision- language adapters

    Balamurali Murugesan, Julio Silva-Rodriguez, Ismail Ben Ayed, and Jose Dolz. Robust calibration of large vision- language adapters. In European Conference on Computer Vision (ECCV), 2024. 2, 3

    2024

  31. [39]

    Automated flower classification over a large number of classes

    Maria-Elena Nilsback and Andrew Zisserman. Automated flower classification over a large number of classes. In In- dian Conference on Computer Vision, Graphics and Image Processing, 2008. 5

    2008

  32. [40]

    Dusenberry, Linchuan Zhang, Ghassen Jerfel, and Dustin Tran

    Jeremy Nixon, Michael W. Dusenberry, Linchuan Zhang, Ghassen Jerfel, and Dustin Tran. Measuring calibration in deep learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Work- shops, 2019. 2, 6

    2019

  33. [41]

    To- wards calibrated robust fine-tuning of vision-language mod- els

    Changdae Oh, Mijoo Kim, Hyesu Lim, Junhyeok Park, Euiseog Jeong, Zhi-Qi Cheng, and Kyungwoo Song. To- wards calibrated robust fine-tuning of vision-language mod- els. Advances in Neural Information Processing Systems (NeurIPS), 2024. 2 10

    2024

  34. [42]

    Theodore Papamarkou, Maria Skoularidou, Konstantina Palla, Laurence Aitchison, Julyan Arbel, David Dun- son, Maurizio Filippone, Vincent Fortuin, Philipp Hennig, Jos´e Miguel Hern´andez-Lobato, Aliaksandr Hubin, Alexan- der Immer, Theofanis Karaletsos, Mohammad Emtiyaz Khan, A...

    2024

  35. [43]

    Cats and dogs

    Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and CV Jawahar. Cats and dogs. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), page 3498–3505, 2012. 5

    2012

  36. [44]

    Learn- ing transferable visual models from natural language super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. In International Conference on Machine Learning...

    2021

  37. [45]

    Clip for all things zero-shot sketch-based image retrieval, fine- grained or not

    Aneeshan Sain, Ayan Kumar Bhunia, Pinaki Nath Chowd- hury, Subhadeep Koley, Tao Xiang, and Yi-Zhe Song. Clip for all things zero-shot sketch-based image retrieval, fine- grained or not. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , page...

    2023

  38. [46]

    Few-shot adaptation of medical vision-language models

    Fereshteh Shakeri, Yunshi Huang, Julio Silva-Rodr ´ıguez, Houda Bahig, An Tang, Jose Dolz, and Ismail Ben Ayed. Few-shot adaptation of medical vision-language models. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 553–563, 2024. 1

    2024

  39. [47]

    A closer look at the few-shot adaptation of large vision-language models

    Julio Silva-Rodriguez, Sina Hajimiri, Ismail Ben Ayed, and Jose Dolz. A closer look at the few-shot adaptation of large vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 23681–23690, 2024. 1, 2, 3, 5, 6

    2024

  40. [48]

    Ucf101: A dataset of 101 human actions classes from videos in the wild

    Khurram Soomro, Amir Roshan Zamir, and Mubarak Shah. Ucf101: A dataset of 101 human actions classes from videos in the wild. In ArXiv Preprint, 2012. 5

    2012

  41. [49]

    An empirical study into what matters for calibrating vision-language models

    Weijie Tu, Weijian Deng, Dylan Campbell, Stephen Gould, and Tom Gedeon. An empirical study into what matters for calibrating vision-language models. In International Confer- ence on Machine Learning (ICML), 2024. 2

    2024

  42. [50]

    Probvlm: Probabilistic adapter for frozen vison-language models

    Uddeshya Upadhyay, Shyamgopal Karthik, Massimiliano Mancini, and Zeynep Akata. Probvlm: Probabilistic adapter for frozen vison-language models. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 1899–1910, 2023. 2, 3

    1910

  43. [51]

    Sclip: Rethink- ing self-attention for dense vision-language inference

    Feng Wang, Jieru Mei, and Alan Yuille. Sclip: Rethink- ing self-attention for dense vision-language inference. In European Conference on Computer Vision, pages 315–332. Springer, 2025. 1

    2025

  44. [52]

    Open-vocabulary calibration for fine-tuned clip

    Shuoyuan Wang, Jindong Wang, Guoqing Wang, Bob Zhang, Kaiyang Zhou, and Hongxin Wei. Open-vocabulary calibration for fine-tuned clip. In Forty-first International Conference on Machine Learning, 2024. 2

    2024

  45. [53]

    Bayesian example selection improves in-context learning for speech, text, and visual modalities

    Siyin Wang, Chao-Han Huck Yang, Ji Wu, and Chao Zhang. Bayesian example selection improves in-context learning for speech, text, and visual modalities. arXiv preprint arXiv:2404.14716, 2024. 3

    2024 arXiv

  46. [54]

    Bayesian deep learn- ing and a probabilistic perspective of generalization

    Andrew G Wilson and Pavel Izmailov. Bayesian deep learn- ing and a probabilistic perspective of generalization. In Advances in Neural Information Processing Systems , pages 4697–4708. Curran Associates, Inc., 2020. 4

    2020

  47. [55]

    Confidence-aware contrastive learn- ing for selective classification

    Yu-Chang Wu, Shen-Huan Lyu, Haopu Shang, Xiangyu Wang, and Chao Qian. Confidence-aware contrastive learn- ing for selective classification. In International Conference on Machine Learning, 2024. 6

    2024

  48. [56]

    Understanding why label smoothing de- grades selective classification and how to fix it

    Guoxuan Xia, Olivier Laurent, Gianni Franchi, and Christos- Savvas Bouganis. Understanding why label smoothing de- grades selective classification and how to fix it. arXiv preprint arXiv:2403.14715, 2024. 6

    2024 arXiv

  49. [57]

    Ehinger, Aude Oliva, and Antonio Torralba

    Jianxiong Xiao, James Hays, Krista A. Ehinger, Aude Oliva, and Antonio Torralba. Sun database: Large-scale scene recognition from abbey to zoo. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3485–3492, 2010. 5

    2010

  50. [58]

    C-tpt: Calibrated test-time prompt tuning for vision-language mod- els via text feature dispersion

    Hee Suk Yoon, Eunseop Yoon, Joshua Tian Jin Tee, Mark A Hasegawa-Johnson, Yingzhen Li, and Chang D Yoo. C-tpt: Calibrated test-time prompt tuning for vision-language mod- els via text feature dispersion. In The Twelfth International Conference on Learning Representations, 2024. 2

    2024

  51. [59]

    Task residual for tuning vision-language models

    Tao Yu, Zhihe Lu, Xin Jin, Zhibo Chen, and Xinchao Wang. Task residual for tuning vision-language models. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10899–10909, 2023. 1, 2, 5

    2023

  52. [60]

    Advances in variational inference

    Cheng Zhang, Judith B ¨utepage, Hedvig Kjellstr ¨om, and Stephan Mandt. Advances in variational inference. IEEE transactions on pattern analysis and machine intelligence , 41(8):2008–2026, 2018. 4

    2008

  53. [61]

    Tip-adapter: Training-free clip-adapter for better vision- language modeling

    Renrui Zhang, Rongyao Fang, Wei Zhang, Peng Gao, Kunchang Li, Jifeng Dai, Yu Qiao, and Hongsheng Li. Tip-adapter: Training-free clip-adapter for better vision- language modeling. In European Conference on Computer Vision (ECCV), pages 1–19, 2022. 1, 2, 5

    2022

  54. [62]

    Conditional prompt learning for vision-language mod- els

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Conditional prompt learning for vision-language mod- els. In Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition , pages 16816–16825,

  55. [63]

    Learning to prompt for vision-language models

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to prompt for vision-language models. Inter- national Journal of Computer Vision (IJCV), 2022. 1, 5

    2022

  56. [64]

    Prompt-aligned gradient for prompt tuning

    Beier Zhu, Yulei Niu, Yucheng Han, Yue Wu, and Hanwang Zhang. Prompt-aligned gradient for prompt tuning. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 15659–15669, 2023. 2

    2023

  57. [65]

    Bayesian cross-modal alignment learning for few-shot out- of-distribution generalization

    Lin Zhu, Xinbing Wang, Chenghu Zhou, and Nanyang Ye. Bayesian cross-modal alignment learning for few-shot out- of-distribution generalization. In Proceedings of the AAAI Conference on Artificial Intelligence , pages 11461–11469,

  58. [2023]

    Theoretical derivations A.1

    3 11 BayesAdapter: enhanced uncertainty estimation in CLIP few-shot adaptation Supplementary Material A. Theoretical derivations A.1. Proposition 1 Here we justify step by step the equivalence stated in Propo- sition 1, by which performing MAP inference in the novel BayesAdapt...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.