REVIEW 4 major objections 6 minor 32 references
Enhancing Few-Shot Out-of-Distribution Detection with Gradient Aligned Context Optimization
T0 review · 4 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read The paper claims that projecting the in-distribution gradient away from the OOD-regularization gradient removes the conflict in few-shot prompt tuning.
desk verdict Straightforward application of a known gradient-projection rule (ProGrad/PCGrad) to LoCoOp, with decent FPR95 gains but a mechanism story that doesn't survive close reading. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the gradient projection rule of Eq. (4), applied to the context vectors of the prompt. The rule treats the OOD regularization gradient $\mathbf{G}_o$ as a fixed reference direction and modifies only the ID gradient: an acute dot product means keep $\mathbf{G}_i$ unchanged, while an obtuse dot product means replace $\mathbf{G}_i$ by its projection onto the subspace orthogonal to $\mathbf{G}_o$. This projection is what lets the two losses cooperate without reweighting them.
What would settle it
Run GaCoOp with the projection branch always disabled and with it always enabled, and record the fraction of training steps where $\mathbf{G}_i \cdot \mathbf{G}_o < 0$ actually occurs. If removing the projection leaves the final FPR95 and ID accuracy unchanged, the conflict it targets was not the limiting factor; if the conflicting fraction is near zero, the projection is not the source of the gain.
Extended reading notes
Core claim
The central claim is embodied in Eq. (4): the update direction is $\mathbf{G}_i$ when $\mathbf{G}_i \cdot \mathbf{G}_o \ge 0$, and otherwise $\mathbf{G}_i - (\mathbf{G}_i \cdot \mathbf{G}_o / \|\mathbf{G}_o\|^2)\,\mathbf{G}_o$. The paper's way of stating it is that the in-distribution gradient decomposes into a non-conflicting orthogonal part and a potentially conflicting parallel part; when the parallel part points against $\mathbf{G}_o$, dropping it prevents OOD regularization from corrupting ID classification. The authors claim this gradient aligned context optimization improves OOD detection on the ImageNet benchmarks and improves ID classification accuracy on ImageNet over CoOp.
Load-bearing premise
The load-bearing premise is that the angle between the two gradient vectors at a single step is a complete and trustworthy measure of conflict, so removing the component of the in-distribution gradient that points against the OOD gradient always helps and never hurts.
Editorial extensions
If this is right
- On the ImageNet OOD benchmark, the one-shot average FPR95 drops from 40.17 for LoCoOp to 29.59 for GaCoOp, and the four-shot average drops from 36.95 to 29.40.
- ID classification accuracy on ImageNet-1K reaches 69.63 percent, slightly above CoOp's 69.38 percent, while training takes only about 30 minutes longer than CoOp.
- The same projection rule transfers to a ResNet-50 backbone, improving average FPR95 from 45.62 for LoCoOp to 33.74 and average AUROC to 91.62.
- Because the projection only changes the update direction on conflicting steps, the rule can be grafted onto any prompt-tuning objective of the form $L_{\text{coop}} + \lambda L_{\text{ood}}$.
Reading between the lines
- The paper does not report how often the conflict condition $\mathbf{G}_i \cdot \mathbf{G}_o < 0$ actually fires; if that fraction is low, the reported gains may come from the unchanged $\mathbf{G}_i$ updates rather than from the projection, a distinction a follow-up could test.
- The angle-only projection ignores gradient magnitudes and curvature, so the same rule may need a step-size correction or a margin on the dot product when applied to larger prompts or to other two-loss objectives.
- The paper's closing belief that an unbiased classifier is also a good OOD detector implies a calibration claim: GaCoOp should also reduce in-distribution overconfidence, which could be checked with expected calibration error on ImageNet.
- The same decomposition could apply to any prompt-tuning setup that combines a task loss with a regularizer, for example supervised contrastive learning plus entropy maximization.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This manuscript proposes GaCoOp, a prompt-tuning method for few-shot out-of-distribution (OOD) detection with CLIP. It builds on LoCoOp's two-term objective (3), which combines an ID cross-entropy classification loss with an OOD entropy regularization loss, and modifies the update rule so that at each step the ID classification gradient Gi is either used directly when Gi·Go ≥ 0 or projected onto the orthogonal complement of the OOD regularization gradient Go otherwise, as stated in Eq. (4). The authors evaluate on ImageNet-1K as ID data and four OOD datasets (iNaturalist, SUN, Places, Texture) under one-shot and four-shot settings with CLIP ViT-B/16 and ResNet-50, reporting FPR95, AUROC, ID classification accuracy, and training time. They report that GaCoOp outperforms LoCoOp on FPR95, improves ID accuracy over CoOp, and is substantially faster than ID-like.
Significance. The proposed update rule is simple and computationally cheap, and the FPR95 improvements over LoCoOp in Table I are consistent across datasets; the training-time advantage over ID-like (1h48m vs. 23h51m in Table II) is practically meaningful. If the mechanism were properly supported, the paper would be a useful incremental contribution to few-shot OOD prompt tuning. However, the current manuscript does not provide direct evidence for the claimed gradient-conflict-mitigation mechanism, and the update in Eq. (4) does not actually optimize the OOD regularization term as part of a joint objective. The empirical evaluation also lacks error bars and multiple seeds, leaving the central claim unsupported in its current form.
major comments (4)
- [III-B, Eq. (4)] The proposed update does not implement optimization of the objective in Eq. (3). In both branches of Eq. (4), Ggacoop is a function of Gi only; Go is used only as a reference direction for deciding whether to project. Consequently, the OOD regularization gradient never enters the update as an optimization direction. The first-order change of Lood along the update is Go·Ggacoop, which is Gi·Go in the acute branch (nonnegative, so Lood tends to increase) and 0 in the obtuse branch. Thus GaCoOp never descends on Lood. This is inconsistent with the paper's claim that it alleviates the conflict by jointly optimizing ID classification and OOD regularization. The authors should either reframe the method as a constrained modification of CoOp in which the OOD term acts only as a projection reference and compare against simply removing λLood from Eq. (3), or provide evidence that the projection improves OOD detection through a different mechanism.
- [IV-C and Table I] The paper reports no ablation or diagnostic that directly supports the gradient-conflict mechanism. There are no statistics on how often the obtuse branch triggers, no distribution of Gi·Go during training, no trajectory of Lood or Lcoop, and no comparison to LoCoOp without the OOD regularization term. Without such evidence, the observed FPR95 improvements could be explained by the projection acting as an ad hoc regularizer of the ID gradient, or by the method partially ignoring a potentially harmful OOD term, rather than by alignment of two co-optimized losses. This missing experiment is load-bearing for the paper's central claim.
- [IV-A, Tables I and II] All reported results are single-run point estimates with no error bars, multiple seeds, or significance tests. Prompt tuning is stochastic due to prompt initialization, batch sampling, and data shuffling, and many reported differences (e.g., ID accuracy 69.63 vs. 69.38 in Table II, AUROC differences around one percentage point in Table I) are likely within seed variance. The authors should report mean ± standard deviation over at least 3–5 seeds and, ideally, paired tests for the key comparison against LoCoOp.
- [III-B] The claim that the orthogonal projection 'can not override the Gi' and that the resulting direction is 'non-conflicting' is not justified as stated. Orthogonality to Go at a single step only ensures zero first-order change of Lood along the update; since both Gi and Go are functions of the current prompt parameters and change after every step, a direction that is orthogonal at step t can become conflicting at step t+1. The paper provides no convergence or safety analysis, and the premise that Go is a trustworthy reference direction is not examined beyond the final metrics.
minor comments (6)
- [Abstract and Section IV-B] The phrases 'inner ID samples' and 'ImageNed-1k' appear to be typos; please use 'in-distribution ID samples' and 'ImageNet-1K'.
- [III-A, Eq. (2)] The symbol K is used in 'K ID-irrelevant region indices' but never defined, and pj should be defined precisely, e.g., as the softmax probability over the j-th region.
- [Fig. 1] The caption for (b) says 'If Gi is aligned with Go', but the condition in Eq. (4) is Gi·Go ≥ 0, which also includes orthogonal directions; the caption and the surrounding text should be consistent on the angle condition.
- [Table III] The table caption does not state the number of shots; the text mentions four-shot results, but the caption should be self-contained, and it should be clear that MCM is a zero-shot method.
- [Abstract] The repository link is provided, but no code is currently released; please make the code available or state the intended release date for reproducibility.
- [I and III-B] The projection rule is described as 'inspired by [27]', but the methodological overlap with Prompt-Aligned Gradient is substantial; the authors should state explicitly what is new relative to [27] beyond the application to OOD detection with LoCoOp.
Circularity Check
No significant circularity: Eq. (4) is an explicit optimization heuristic, and the reported OOD/ID gains are empirical comparisons, not fitted predictions.
full rationale
The paper's central update rule, Eq. (4), is not derived from a fitted parameter, a self-citation chain, or a renamed empirical pattern. Gi and Go are defined as gradients of the two losses in Eq. (3), and the rule either keeps Gi or projects it orthogonal to Go, following the gradient-projection idea cited to reference [27], which is by different authors. The claimed improvements in OOD detection and ID classification are supported by Tables I–III, where the same hyperparameters as LoCoOp are used and the results are compared against external baselines. The only self-citation is reference [12] (K. Song is a coauthor), used to motivate that CLIP bias can cause conflict between ID and OOD objectives; that motivational premise is not load-bearing for the construction of Eq. (4) and does not force any experimental outcome. The statement that the conflict is 'mitigated' is, at most, a design property of the update—the chosen direction always has nonnegative dot product with Go—but this is not a prediction extracted from data, and the empirical contribution is independent of this framing. No circular derivation step is present.
Assumptions & free parameters
free parameters (5)
- OOD regularization weight lambda =
not reported (inherited from LoCoOp)
- learning rate =
0.002
- training epochs =
50
- batch size =
32
- number of prompt tokens =
16
assumptions (4)
- domain assumption The conflict between ID classification and OOD regularization is fully captured by the sign of the dot product Gi dot Go.
- domain assumption Background regions of ID images, as identified by CLIP's attention, are valid surrogate OOD samples for L_ood.
- ad hoc to paper Projecting Gi onto the orthogonal complement of Go when Gi dot Go < 0 preserves ID classification and improves OOD detection.
- standard math Orthogonal projection removes the conflicting component of a vector (linear algebra property).
Cite this review
Pith. "Pith review of Enhancing Few-Shot Out-of-Distribution Detection with Gradient Aligned Context Optimization." pith.science (2026). https://pith.science/paper/I74O3ACI
@misc{pith2026241115736,
author = {Pith},
title = {Pith review of: Enhancing Few-Shot Out-of-Distribution Detection with Gradient Aligned Context Optimization},
year = {2026},
howpublished = {\url{https://pith.science/paper/I74O3ACI}},
note = {Machine review of arXiv:2411.15736}
}
read the original abstract
Few-shot out-of-distribution (OOD) detection aims to detect OOD images from unseen classes with only a few labeled in-distribution (ID) images. To detect OOD images and classify ID samples, prior methods have been proposed by regarding the background regions of ID samples as the OOD knowledge and performing OOD regularization and ID classification optimization. However, the gradient conflict still exists between ID classification optimization and OOD regularization caused by biased recognition. To address this issue, we present Gradient Aligned Context Optimization (GaCoOp) to mitigate this gradient conflict. Specifically, we decompose the optimization gradient to identify the scenario when the conflict occurs. Then we alleviate the conflict in inner ID samples and optimize the prompts via leveraging gradient projection. Extensive experiments over the large-scale ImageNet OOD detection benchmark demonstrate that our GaCoOp can effectively mitigate the conflict and achieve great performance. Code will be available at https://github.com/BaoshunWq/ood-GaCoOp.
Figures
Reference graph
Works this paper leans on
-
[27]
Prompt-aligned gradient for prompt tuning,
B. Zhu, Y . Niu, Y . Han, Y . Wu, and H. Zhang, “Prompt-aligned gradient for prompt tuning,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2023, pp. 15 659–15 669
2023
-
[1]
Clipood: Generalizing clip to out-of-distributions,
Y . Shu, X. Guo, J. Wu, X. Wang, J. Wang, and M. Long, “Clipood: Generalizing clip to out-of-distributions,” in International Conference on Machine Learning . PMLR, 2023, pp. 31 716–31 731
work page 2023
-
[2]
A baseline for detecting misclassified and out-of-distribution examples in neural networks,
D. Hendrycks and K. Gimpel, “A baseline for detecting misclassified and out-of-distribution examples in neural networks,” arXiv preprint arXiv:1610.02136, 2016
arXiv 2016
-
[3]
Learning transferable visual models from natural language supervision,
A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark et al., “Learning transferable visual models from natural language supervision,” in International conference on machine learning . PMLR, 2021, pp. 8748–8763
2021
-
[4]
Delving into out-of- distribution detection with vision-language representations,
Y . Ming, Z. Cai, J. Gu, Y . Sun, W. Li, and Y . Li, “Delving into out-of- distribution detection with vision-language representations,” Advances in neural information processing systems , vol. 35, pp. 35 087–35 102, 2022
2022
-
[5]
Enhancing the reliability of out-of- distribution image detection in neural networks,
S. Liang, Y . Li, and R. Srikant, “Enhancing the reliability of out-of- distribution image detection in neural networks,” in 6th International Conference on Learning Representations, ICLR 2018 , 2018
work page 2018
-
[6]
Vim: Out-of-distribution with virtual-logit matching,
H. Wang, Z. Li, L. Feng, and W. Zhang, “Vim: Out-of-distribution with virtual-logit matching,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2022, pp. 4921–4930
work page 2022
-
[7]
Out-of-distribution detection with deep nearest neighbors,
Y . Sun, Y . Ming, X. Zhu, and Y . Li, “Out-of-distribution detection with deep nearest neighbors,” in International Conference on Machine Learning. PMLR, 2022, pp. 20 827–20 840
work page 2022
Show all 32 references
-
[8]
Non-parametric outlier synthesis,
L. Tao, X. Du, J. Zhu, and Y . Li, “Non-parametric outlier synthesis,” in The Eleventh International Conference on Learning Representations , 2023
2023
-
[9]
Locoop: Few-shot out- of-distribution detection via prompt learning,
A. Miyai, Q. Yu, G. Irie, and K. Aizawa, “Locoop: Few-shot out- of-distribution detection via prompt learning,” Advances in Neural Information Processing Systems , vol. 36, 2024
2024
-
[10]
Id-like prompt learning for few-shot out-of-distribution detection,
Y . Bai, Z. Han, B. Cao, X. Jiang, Q. Hu, and C. Zhang, “Id-like prompt learning for few-shot out-of-distribution detection,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 17 480–17 489
2024
-
[11]
Invariant learning via probability of sufficient and necessary causes,
M. Yang, Z. Fang, Y . Zhang, Y . Du, F. Liu, J.-F. Ton, J. Wang, and J. Wang, “Invariant learning via probability of sufficient and necessary causes,” Advances in Neural Information Processing Systems , vol. 36, pp. 79 832–79 857, 2023
2023
-
[12]
Fd-align: feature discrimination alignment for fine-tuning pre-trained models in few-shot learning,
K. Song, H. Ma, B. Zou, H. Zhang, and W. Huang, “Fd-align: feature discrimination alignment for fine-tuning pre-trained models in few-shot learning,” Advances in Neural Information Processing Systems , vol. 36, 2024
2024
-
[13]
Learn to rectify the bias of clip for unsupervised semantic segmentation,
J. Wang and G. Kang, “Learn to rectify the bias of clip for unsupervised semantic segmentation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 4102–4112
2024
-
[14]
Visualbert: A simple and performant baseline for vision and language,
L. H. Li, M. Yatskar, D. Yin, C.-J. Hsieh, and K.-W. Chang, “Visualbert: A simple and performant baseline for vision and language,” arXiv preprint arXiv:1908.03557, 2019
1908 arXiv
-
[15]
Vilt: Vision-and-language transformer without convolution or region supervision,
W. Kim, B. Son, and I. Kim, “Vilt: Vision-and-language transformer without convolution or region supervision,” in International conference on machine learning . PMLR, 2021, pp. 5583–5594
2021
-
[16]
Filip: Fine-grained interactive language-image pre-training,
L. Yao, R. Huang, L. Hou, G. Lu, M. Niu, H. Xu, X. Liang, Z. Li, X. Jiang, and C. Xu, “Filip: Fine-grained interactive language-image pre-training,” in International Conference on Learning Representations , 2021
2021
-
[17]
Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation,
J. Li, D. Li, C. Xiong, and S. Hoi, “Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation,” in International conference on machine learning . PMLR, 2022, pp. 12 888–12 900
2022
-
[18]
Learning to prompt for vision- language models,
K. Zhou, J. Yang, C. C. Loy, and Z. Liu, “Learning to prompt for vision- language models,” International Journal of Computer Vision , vol. 130, no. 9, pp. 2337–2348, 2022
2022
-
[19]
Visual prompt tuning,
M. Jia, L. Tang, B.-C. Chen, C. Cardie, S. Belongie, B. Hariharan, and S.-N. Lim, “Visual prompt tuning,” in European Conference on Computer Vision. Springer, 2022, pp. 709–727
2022
-
[20]
Maple: Multi-modal prompt learning,
M. U. Khattak, H. Rasheed, M. Maaz, S. Khan, and F. S. Khan, “Maple: Multi-modal prompt learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 19 113–19 122
2023
-
[21]
What does a platypus look like? generating customized prompts for zero-shot image classification,
S. Pratt, I. Covert, R. Liu, and A. Farhadi, “What does a platypus look like? generating customized prompts for zero-shot image classification,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 15 691–15 701
2023
-
[22]
Exploring the limits of out- of-distribution detection,
S. Fort, J. Ren, and B. Lakshminarayanan, “Exploring the limits of out- of-distribution detection,” Advances in Neural Information Processing Systems, vol. 34, pp. 7068–7081, 2021
2021
-
[23]
V os: Learning what you don’t know by virtual outlier synthesis,
X. Du, Z. Wang, M. Cai, and Y . Li, “V os: Learning what you don’t know by virtual outlier synthesis,” in International Conference on Learning Representations, 2022
2022
-
[24]
Amu-tuning: Effective logit bias for clip-based few-shot learning,
Y . Tang, Z. Lin, Q. Wang, P. Zhu, and Q. Hu, “Amu-tuning: Effective logit bias for clip-based few-shot learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 23 323–23 333
2024
-
[25]
Extract free dense labels from clip,
C. Zhou, C. C. Loy, and B. Dai, “Extract free dense labels from clip,” in European Conference on Computer Vision. Springer, 2022, pp. 696– 712
2022
-
[26]
Universal domain adaptation through self supervision,
K. Saito, D. Kim, S. Sclaroff, and K. Saenko, “Universal domain adaptation through self supervision,” Advances in neural information processing systems, vol. 33, pp. 16 282–16 292, 2020
2020
-
[28]
Imagenet: A large-scale hierarchical image database,
J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “Imagenet: A large-scale hierarchical image database,” in 2009 IEEE conference on computer vision and pattern recognition . Ieee, 2009, pp. 248–255
2009
-
[29]
The inaturalist species classifi- cation and detection dataset,
G. Van Horn, O. Mac Aodha, Y . Song, Y . Cui, C. Sun, A. Shepard, H. Adam, P. Perona, and S. Belongie, “The inaturalist species classifi- cation and detection dataset,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2018, pp. 8769–8778
2018
-
[30]
Sun database: Large-scale scene recognition from abbey to zoo,
J. Xiao, J. Hays, K. A. Ehinger, A. Oliva, and A. Torralba, “Sun database: Large-scale scene recognition from abbey to zoo,” in 2010 IEEE computer society conference on computer vision and pattern recognition. IEEE, 2010, pp. 3485–3492
2010
-
[31]
Places: A 10 million image database for scene recognition,
B. Zhou, A. Lapedriza, A. Khosla, A. Oliva, and A. Torralba, “Places: A 10 million image database for scene recognition,” IEEE transactions on pattern analysis and machine intelligence, vol. 40, no. 6, pp. 1452–1464, 2017
2017
-
[32]
Describing textures in the wild,
M. Cimpoi, S. Maji, I. Kokkinos, S. Mohamed, and A. Vedaldi, “Describing textures in the wild,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2014, pp. 3606–3613
2014
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.