Pith. sign in

REVIEW 2 major objections 2 minor 29 references

GP-Adapter: Gaussian Process CLIP-Adapter for Few-Shot Out-of-Distribution Detection

T0 review · 2 major / 2 minor · reviewed 2026-06-27 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read GP-Adapter adds class-wise Gaussian processes to frozen CLIP embeddings to produce variance-aware scores for few-shot out-of-distribution detection.

desk verdict GP-Adapter adds modality-specific one-class GPs on frozen CLIP embeddings for a training-free variance score in few-shot OOD, but the abstract leaves the actual separation quality and ablation results unshown. read the letter →

arxiv 2606.07102 v1 pith:HBJNGYCN submitted 2026-06-05 cs.CV cs.AI

Taisei Saito , Koretaka Ogata , Takafumi Hiroi This is my paper · ORCID
classification cs.CVcs.AI
keywords few-shotclassificationout-of-distributiondetectionGaussianprocessesCLIPadapteruncertaintyestimationvision-languagemodelstraining-freeadaptation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces a training-free method that places one-class Gaussian processes on top of CLIP image and text embeddings to supply uncertainty estimates where standard similarity scores provide none. Separate GPs are fit per class using an RBF kernel on images and a linear kernel on prompts, then their predictive variances are fused into a single confidence measure. This construction requires only a small cache of K-shot examples per class and no updates to the underlying CLIP weights. A reader would care because the resulting variance signal improves detection of samples from unseen distributions while remaining competitive on standard few-shot classification tasks. The experiments further indicate that the uncertainty signal complements existing prompt-tuning techniques rather than replacing them.

What carries the argument

Modality-specific class-wise one-class Gaussian processes whose predictive variances are fused into a single OOD confidence score.

What would settle it

An OOD benchmark in which the AUROC or FPR95 of the fused GP variance score is no better than, or worse than, the deterministic CLIP similarity baseline.

Watch

Extended reading notes

Core claim

GP-Adapter constructs modality-specific, class-wise one-class GPs on top of frozen CLIP embeddings using an RBF kernel for image features and a linear kernel for text prompts and fuses their predictive statistics to produce a variance-aware confidence score for OOD detection.

Load-bearing premise

The fused predictive variance from the class-wise GPs reliably separates in-distribution examples from out-of-distribution examples on the tested benchmarks.

Editorial extensions

If this is right

  • Competitive accuracy on ImageNet few-shot classification using only the cached examples.
  • Consistent gains in OOD detection metrics when the GP variance is added to prompt-learning baselines.
  • Memory requirement scales as O(C K squared) where C is the number of classes and K the shots per class.
  • The method works without any gradient updates to the CLIP backbone.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same variance-fusion idea could be applied to other frozen vision-language backbones by swapping only the embedding extractor.
  • In safety-critical low-data settings the explicit uncertainty signal may reduce over-confident errors on shifted inputs.
  • Replacing the fixed RBF and linear kernels with a learned or adaptive kernel choice might further tighten the separation between in- and out-of-distribution variances.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 2 minor

Summary. The paper proposes GP-Adapter, a training-free framework augmenting frozen CLIP with class-wise one-class Gaussian Processes (RBF kernel on image embeddings, linear kernel on text prompts) whose predictive variances are fused into a variance-aware confidence score for few-shot classification and OOD detection. It reports competitive few-shot accuracy on ImageNet and consistent OOD gains when combined with prompt-learning baselines, with memory scaling O(CK^2) and code released.

Significance. If the central empirical claim holds, the work supplies a lightweight, non-fine-tuning route to uncertainty quantification on CLIP embeddings that complements prompt learning, with explicit reproducibility via code release and a clear complexity bound. This could be useful for low-data reliability under distribution shift.

major comments (2)
  1. [Method] The central OOD claim rests on the fused class-wise GP predictive variance being reliably higher for OOD than ID inputs, yet no derivation or monotonicity argument is supplied showing that the fusion (max/product/learned) of one-class GP variances separates distributional distance in CLIP space; this is load-bearing for attributing gains to the GP term rather than the prompt baseline.
  2. [Experiments] The hyperparameter selection for the RBF length-scale and noise variance is described only as 'lightweight'; without an explicit ID-only validation protocol stated, it is unclear whether the reported OOD improvements could be driven by implicit leakage or baseline effects rather than the uncertainty model.
minor comments (2)
  1. [Abstract] The abstract asserts 'competitive few-shot performance' and 'consistent improvements' but supplies no numerical values, tables, or error bars; these should be summarized with at least one key metric (e.g., AUROC or accuracy delta) for immediate readability.
  2. [Method] Notation for the fusion rule and the exact form of the variance-aware score is introduced without an equation number; adding an explicit equation would clarify how image and text GP outputs are combined.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments and the opportunity to clarify our work. Below we respond point-by-point to the major comments and indicate the revisions we will make.

read point-by-point responses

  1. Referee: [Method] The central OOD claim rests on the fused class-wise GP predictive variance being reliably higher for OOD than ID inputs, yet no derivation or monotonicity argument is supplied showing that the fusion (max/product/learned) of one-class GP variances separates distributional distance in CLIP space; this is load-bearing for attributing gains to the GP term rather than the prompt baseline.

    Authors: We agree that a formal derivation of monotonicity for the variance fusion is absent. GP-Adapter is an empirical framework that builds on the standard property of one-class GPs: predictive variance increases with distance from the inducing points in the embedding space. The class-wise one-class GPs (RBF on images, linear on text) are intended to capture this effect per class, and the fusion (max, product, or learned) is chosen to produce a scalar uncertainty signal. While we do not claim a theoretical guarantee that the fused variance strictly separates ID from OOD in CLIP space, the reported experiments show consistent OOD AUROC/AUPR gains when the GP term is added to prompt-learning baselines. In revision we will expand the method section with additional intuition, a small illustrative 2-D example of variance behavior, and explicit caveats that the separation is demonstrated empirically rather than proven. We will also emphasize that improvements are measured relative to the same prompt baseline, so the incremental benefit is attributable to the added GP variance. revision: partial

  2. Referee: [Experiments] The hyperparameter selection for the RBF length-scale and noise variance is described only as 'lightweight'; without an explicit ID-only validation protocol stated, it is unclear whether the reported OOD improvements could be driven by implicit leakage or baseline effects rather than the uncertainty model.

    Authors: We thank the referee for highlighting this ambiguity. The length-scale and noise variance are selected via a small held-out subset of the ID few-shot examples (typically 20 % of the K-shot cache per class) using a simple validation criterion on ID classification accuracy; no OOD data is ever used. We will revise the experimental setup and hyperparameter section to state this ID-only protocol explicitly, including the exact split and selection criterion, so that readers can verify the absence of leakage. revision: yes

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity detected in derivation chain

full rationale

The paper describes a training-free method that fits standard class-wise one-class GPs (RBF kernel on image embeddings, linear kernel on text) to a small K-shot cache of frozen CLIP features and then computes the usual GP predictive variance on new inputs before fusing those variances into a confidence score. No equation or claim reduces any reported performance quantity to a fitted parameter by construction, no self-citations are invoked as load-bearing premises, and no uniqueness theorem or ansatz is smuggled in. The central empirical claim (competitive few-shot accuracy plus OOD gains when combined with prompt learning) rests on external benchmark results rather than an internal redefinition of the input data. The derivation is therefore self-contained and externally falsifiable.

Assumptions & free parameters 1 free parameters · 1 assumptions · 0 invented entities

The method rests on the suitability of CLIP embeddings for GP regression and on the choice of standard kernels; no new physical entities are postulated.

free parameters (1)
  • GP kernel hyperparameters
    Lightweight hyperparameter selection is required for the RBF and linear kernels; values are not stated in the abstract.
assumptions (1)
  • domain assumption CLIP embeddings form a feature space in which Gaussian Process predictive variance correlates with distributional shift
    Invoked when the method treats GP variance as an OOD indicator without further justification in the abstract.

how reviews work

0 comments
Cite this review

Pith. "Pith review of GP-Adapter: Gaussian Process CLIP-Adapter for Few-Shot Out-of-Distribution Detection." pith.science (2026). https://pith.science/paper/HBJNGYCN

@misc{pith2026260607102,
  author       = {Pith},
  title        = {Pith review of: GP-Adapter: Gaussian Process CLIP-Adapter for Few-Shot Out-of-Distribution Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HBJNGYCN}},
  note         = {Machine review of arXiv:2606.07102}
}
abstract

We propose GP-Adapter, a training-free framework that augments CLIP (Contrastive Language-Image Pre-training) with Gaussian Process (GP) uncertainty modeling for few-shot classification and out-of-distribution (OOD) detection. While CLIP achieves strong zero-shot recognition, it yields deterministic similarity scores and offers limited uncertainty information, which is critical under distribution shift and data scarcity. GP-Adapter constructs modality-specific, class-wise one-class GPs on top of frozen CLIP embeddings using an RBF kernel for image features and a linear kernel for text prompts and fuses their predictive statistics to produce a variance-aware confidence score for OOD detection. The method requires no fine-tuning of the CLIP backbone and relies only on a small $K$-shot cache and lightweight hyperparameter selection, with memory cost scaling as $O(CK^2)$ for $C$ classes and $K$ shots. Experiments on ImageNet and multiple OOD benchmarks show that GP-Adapter provides competitive few-shot performance and consistently improves OOD detection when combined with prompt-learning baselines, highlighting the complementarity between GP-based uncertainty modeling and prompt learning. Overall, our results suggest that integrating probabilistic inference with large pre-trained vision-language models can improve reliability in low-data and distribution-shifted settings. Code is available at https://github.com/tms-byte/GP-Adapter

Figures

Figures reproduced from arXiv: 2606.07102 by the authors.

Figure 1
Figure 1. t-SNE visualization of CLIP embeddings. Image and [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of GP-Adapter with one-class Gaussian Processes. Frozen CLIP encoders produce image embeddings from a small K-shot support set and a query image, and text embeddings from class-specific text prompts. For each class, modality￾specific one-class Gaussian Processes output predictive means and variances. These are fused via a weighted combination and converted into OOD scores using the proposed variance-aware M… view at source ↗
Figure 3
Figure 3. Range of GP predictive variances across classes im [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (1 more)
Figure 5
Figure 5. Figure 5: Effect of the number of shots per class on OOD [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

29 extracted references · 2 canonical work pages

  1. [1]

    Out-of- distribution detection: A task-oriented survey of recent advances,

    S. Lu, Y . Wang, L. Sheng, L. He, A. Zheng, and J. Liang, “Out-of- distribution detection: A task-oriented survey of recent advances,”ACM Comput. Surv., 2025

    2025

  2. [2]

    Dropout as a bayesian approximation: Representing model uncertainty in deep learning,

    Y . Gal and Z. Ghahramani, “Dropout as a bayesian approximation: Representing model uncertainty in deep learning,” inProceedings of The 33rd International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, vol. 48, 2016, pp. 1050–1059

    2016

  3. [3]

    Learning transferable visual models from natural language supervi- sion,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever, “Learning transferable visual models from natural language supervi- sion,” inProceedings of the 38th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, vol. 139, 2021, pp. 8748–8763

    2021

  4. [4]

    Learning to prompt for vision- language models,

    K. Zhou, J. Yang, C. C. Loy, and Z. Liu, “Learning to prompt for vision- language models,”International Journal of Computer Vision, vol. 130, no. 9, pp. 2337–2348, 2022

    2022

  5. [5]

    Locoop: Few-shot out- of-distribution detection via prompt learning,

    A. Miyai, Q. Yu, G. Irie, and K. Aizawa, “Locoop: Few-shot out- of-distribution detection via prompt learning,” inAdvances in Neural Information Processing Systems, 2023, pp. 76 298–76 310

    2023

  6. [6]

    Clipn for zero-shot ood detection: Teaching clip to say no,

    H. Wang, Y . Li, H. Yao, and X. Li, “Clipn for zero-shot ood detection: Teaching clip to say no,” inProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2023, pp. 1802–1812

    2023

  7. [7]

    C. E. Rasmussen and C. K. I. Williams,Gaussian Processes for Machine Learning. MIT Press, 2006

    2006

  8. [8]

    Uncertainty- aware out-of-distribution detection with gaussian processes,

    Y . Chen, C.-L. Sung, A. Kusari, X. Song, and W. Sun, “Uncertainty- aware out-of-distribution detection with gaussian processes,”arXiv preprint arXiv:2412.20918, 2024

    work page arXiv 2024

Show all 29 references
  1. [9]

    Ima- genet: A large-scale hierarchical image database,

    J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Ima- genet: A large-scale hierarchical image database,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2009, pp. 248–255

    2009

  2. [10]

    A baseline for detecting misclassified and out-of-distribution examples in neural networks,

    D. Hendrycks and K. Gimpel, “A baseline for detecting misclassified and out-of-distribution examples in neural networks,” inProceedings of the 5th International Conference on Learning Representations (ICLR), 2017, poster session

    2017

  3. [11]

    Energy-based out-of- distribution detection,

    W. Liu, X. Wang, J. D. Owens, and Y . Li, “Energy-based out-of- distribution detection,” inAdvances in Neural Information Processing Systems (NeurIPS), vol. 33, 2020

    2020

  4. [12]

    Vim: Out-of-distribution with virtual-logit matching,

    H. Wang, Z. Li, L. Feng, and W. Zhang, “Vim: Out-of-distribution with virtual-logit matching,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 4911– 4920

    2022

  5. [13]

    Out-of-distribution detection with deep nearest neighbors,

    Y . Sun, Y . Ming, X. Zhu, and Y . Li, “Out-of-distribution detection with deep nearest neighbors,” inProceedings of the 39th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, K. Chaudhuri, S. Jegelka, L. Song, C. Szepesvari, G. Niu, ...

    2022

  6. [14]

    Extending binary neural networks to bayesian neural networks with probabilistic interpretation of binary weights,

    T. Saito, K. Ando, and T. Asai, “Extending binary neural networks to bayesian neural networks with probabilistic interpretation of binary weights,”IEICE TRANSACTIONS on Information, vol. E107-D, no. 8, pp. 949–957, 2024

    2024

  7. [15]

    Out-of- distribution data detection using bayesian convolutional neural network with variational inference,

    K. Minagawa, T. Saito, S. Kojima, K. Ando, and T. Asai, “Out-of- distribution data detection using bayesian convolutional neural network with variational inference,” in2024 International Joint Conference on Neural Networks (IJCNN), 2024, pp. 1–8

    2024

  8. [16]

    Delving into out-of-distribution detection with vision-language representations,

    Y . Ming, Z. Cai, J. Gu, Y . Sun, W. Li, and Y . Li, “Delving into out-of-distribution detection with vision-language representations,” in Proceedings of the 36th International Conference on Neural Information Processing Systems, 2022

    2022

  9. [17]

    Gl-mcm: Global and local maximum concept matching for zero-shot out-of-distribution detection,

    A. Miyai, Q. Yu, G. Irie, and K. Aizawa, “Gl-mcm: Global and local maximum concept matching for zero-shot out-of-distribution detection,” International Journal of Computer Vision, vol. 133, no. 12, pp. 3586– 3596, 2025

    2025

  10. [18]

    Tip-adapter: Training-free adaption of clip for few-shot classification,

    R. Zhang, Z. Wei, R. Fang, P. Gao, K. Li, J. Dai, Y . Qiao, and H. Li, “Tip-adapter: Training-free adaption of clip for few-shot classification,” inProceedings of the 17th European Conference on Computer Vision (ECCV), 2022, pp. 493–510

    2022

  11. [19]

    Dual-adapter: Training-free dual adaptation for few-shot out-of-distribution detection,

    X. Chen, Y . Li, and H. Chen, “Dual-adapter: Training-free dual adaptation for few-shot out-of-distribution detection,”arXiv preprint arXiv:2405.16146, 2024

    2024

  12. [20]

    Non-parametric outlier synthesis,

    L. Tao, X. Du, X. Zhu, and Y . Li, “Non-parametric outlier synthesis,” inProceedings of the 11th International Conference on Learning Rep- resentations (ICLR), 2023

    2023

  13. [21]

    Setar: Out-of-distribution detection with selective low-rank approximation,

    Y . Li, B. Xiong, G. Chen, and Y . Chen, “Setar: Out-of-distribution detection with selective low-rank approximation,” inAdvances in Neural Information Processing Systems (NeurIPS), 2024

    2024

  14. [22]

    Exploring the limits of out- of-distribution detection,

    S. Fort, J. Ren, and B. Lakshminarayanan, “Exploring the limits of out- of-distribution detection,” inAdvances in Neural Information Processing Systems (NeurIPS), 2021, pp. 7068–7081

    2021

  15. [23]

    Enhancing the reliability of out-of- distribution image detection in neural networks,

    S. Liang, Y . Li, and R. Srikant, “Enhancing the reliability of out-of- distribution image detection in neural networks,” inProceedings of the 6th International Conference on Learning Representations (ICLR), 2018

    2018

  16. [24]

    The inaturalist species classification and detection dataset,

    G. Van Horn, O. Mac Aodha, Y . Song, A. Shepard, H. Adam, P. Perona, and S. Belongie, “The inaturalist species classification and detection dataset,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018, pp. 8769–8778

    2018

  17. [25]

    Sun database: Large-scale scene recognition from abbey to zoo,

    J. Xiao, J. Hays, K. A. Ehinger, A. Oliva, and A. Torralba, “Sun database: Large-scale scene recognition from abbey to zoo,”Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3485–3492, 2010

    2010

  18. [26]

    Places: A 10 million image database for scene recognition,

    B. Zhou, A. Lapedriza, A. Khosla, A. Oliva, and A. Torralba, “Places: A 10 million image database for scene recognition,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 40, no. 6, pp. 1452– 1464, 2017

    2017

  19. [27]

    Describing textures in the wild,

    M. Cimpoi, S. Maji, I. Kokkinos, S. Mohamed, and A. Vedaldi, “Describing textures in the wild,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014, pp. 3606– 3613

    2014

  20. [28]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, “An image is worth 16x16 words: Transformers for image recognition at scale,” inProceedings of the 9th International ...

    2021

  21. [29]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, pp. 770–778

    2016

Pith tools

Reviewed June 27, 2026 · model on record in the stance chip above.