Pith. sign in

REVIEW 4 major objections 6 minor 64 references

FDBPL: Faster Distillation-Based Prompt Learning for Region-Aware Vision-Language Models Adaptation

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Distillation-based prompt learning can run 2.2x faster without losing accuracy.

desk verdict A plausible caching idea for distillation-based prompt learning, but the 2.2x speedup likely excludes the dominant offline teacher cost; the accuracy gains are real but modest. read the letter →

arxiv 2505.18053 v1 pith:SQUZU7FM submitted 2025-05-23 cs.CV cs.AI

Zherui Zhang , Jiaxin Wu , Changwei Wang , Rongtao Xu , Longzhao Huang , Wenhao Xu , Wenbo Xu , Li Guo
show 1 more author
Shibiao Xu
This is my paper
classification cs.CVcs.AI
keywords vision-languagemodelspromptlearningknowledgedistillationzero-shotgeneralizationCLIPadaptationtrainingefficiencyregion-awarepromptssoftlabelcaching
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

FDBPL is a prompt-learning method that adapts a frozen CLIP model to downstream tasks by distilling knowledge from a larger CLIP teacher. The paper's central claim is that the teacher's soft supervision can be computed once, stored in a Region Information Lookup table, and reused across every training epoch, removing the repeated online teacher inference that makes earlier distillation-based prompt learning slow. On this basis FDBPL adds a region-aware dual-prompt scheme: positive prompts learn real class semantics from high-information cropped regions, while negative prompts learn to reject low-information regions. The result, the paper reports, is a 2.2x average training speed-up over KDPL with comparable or better zero-shot accuracy, including +1.13 new-class accuracy on ImageNet.

What carries the argument

The central mechanism is the Region Information Lookup (RIL) table: a precomputed store of crop coordinates, augmentation types, and teacher soft labels that lets the student fetch supervision instead of recomputing it. Around it sit two components: Region-Aware Dual-Prompt (RADP) learning, which uses an entropy-derived information weight $w$ to route high-information regions to a positive prompt and low-information regions to a negative prompt; and Prompt-Cascaded Difference (PCD) learning, which forms first-order differences between positive and negative text features and second-order differences between classes, aligning intra-class diversity and inter-class structure. The Top-K label sparsification strategies (MS and MR) keep the table compact enough for fast I/O.

What would settle it

Train FDBPL on ImageNet with the Region Information Lookup table refreshed every few epochs by re-running the teacher on the same crops, and compare new-class accuracy to the one-time-cache version. If fresh supervision improves accuracy by more than the run-to-run variance reported, the cached labels are stale and the 2.2x speed-up trades away some of the claimed generalization gain.

Watch

Extended reading notes

Core claim

The discovery is that the generalization benefit of distillation-based prompt learning does not require the teacher to be invoked during student training. By pre-computing teacher soft labels, spatial coordinates, and augmentation metadata for randomly cropped regions and storing them in the Region Information Lookup table, FDBPL turns distillation into a fast table lookup. Two label-sparsification strategies, Marginal Smoothing with Top-K and Marginal Re-Norm with Top-K, compress stored soft labels so I/O does not become the new bottleneck. The paper further claims that low-information crop regions, normally discarded, can be exploited through a learnable negative prompt that says 'no,' and that a two-level difference space between positive and negative prompts captures intra-class and inter-class structure, improving zero-shot recognition on novel classes.

Load-bearing premise

The frozen teacher's soft labels, computed once before training, remain valid supervision for every student prompt update across all 50 training epochs, so student prompt drift does not make the cached targets stale.

Editorial extensions

If this is right

  • Distillation-based prompt learning can be decoupled from a specific teacher deployment: the table is built offline once, so different or stronger teachers can be swapped in without retraining the student.
  • Training time on large-scale datasets such as ImageNet-1K drops from a KDPL-level slowdown to near-native prompt learning, because per-epoch teacher inference is eliminated.
  • Low-information regions, previously discarded in FKD-style pipelines, become training signal that teaches the student to reject uncertain input, improving novel-class and cross-dataset accuracy.
  • The RIL table's storage and I/O costs scale with the number of classes; Top-K sparsification keeps this manageable and yields an optimal operating point near K=20.
  • FDBPL is compatible with existing prompt-learning bases such as CoOp and PromptSRC, so the acceleration can be layered onto current adaptation pipelines.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The caching design suggests an asynchronous teacher-update protocol: refreshing the table every few epochs with a stronger or ensemble teacher might recover any staleness without giving back the full online-inference cost.
  • The negative-prompt 'say no' behavior is effectively a lightweight out-of-distribution detector; it could be evaluated on open-set recognition benchmarks beyond the robustness tests reported here.
  • Because the table stores teacher outputs rather than model parameters, the method may transfer to other vision-language model families or to tasks such as detection and segmentation where region-level soft supervision is natural.
  • The paper's efficiency claim is wall-clock measured on one GPU setting; the practical speed-up will depend on disk and CPU I/O speed and dataset size, so the 2.2x figure is likely workload-dependent.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. FDBPL proposes an offline pre-computation and caching strategy for distillation-based prompt learning in CLIP adaptation. The method samples M random crops per training image, stores the teacher's soft outputs (with Top-K sparsification) in a Region Information Lookup (RIL) table, and then trains positive and negative learnable prompts on the student CLIP using entropy-weighted KL divergence, a uniform-distribution rejection loss, and first/second-order difference losses. The paper reports base-to-new, cross-dataset, and robustness evaluations over 11 datasets and claims an average 2.2x training speedup relative to KDPL.

Significance. If fully supported, the RIL idea would be useful: it separates expensive teacher inference from prompt optimization and provides a concrete space-for-time trade-off, while the dual positive-negative prompt design is an interesting way to exploit low-information crops instead of discarding them. The paper contains broad benchmark coverage and ablations of loss components, teacher capacity, and label sparsity. However, the significance is currently limited: the headline speedup does not account for the offline RIL construction, the accuracy advantage over existing strong baselines is small and in the aggregate negative relative to PromptSRC, and several central hyperparameters are missing. I do not see a circularity problem in the proposed losses. The concern that cached teacher targets become stale is also not, by itself, a correctness issue, because the teacher model is frozen and its output for a fixed crop is epoch-independent; the real issue is the cost and I/O of building and reading the cache.

major comments (4)
  1. [Sec. 4.1.1 / Fig. 8] The claimed 2.2x faster training is only for the student prompt-training phase, not for the full procedure. Section 4.1.1 states that M=500 random crops are generated per training image for the RIL table; under the 16-shot ImageNet base-to-new setup this is about 16,000 x 500 = 8,000,000 teacher forward passes on crops, while KDPL's online teacher inference over 50 epochs is about 16,000 x 50 = 800,000 teacher forward passes on images. If Figure 8 excludes RIL construction, storage writing, and crop-reading I/O, the reported speedup is not end-to-end and may even reverse on total wall-clock time. Please report end-to-end timing for both methods, state how many of the 500 stored crops are actually consumed per epoch, and tabulate the per-dataset training times that currently appear only in Figure 8.
  2. [Table 2 / Sec. 4.1.2] The abstract's 'superior performance' claim is not supported by the aggregate result: the average harmonic mean in Table 2 is 74.57 for FDBPL but 74.97 for PromptSRC, and the average advantage over KDPL is only 0.58 HM. The paper reports three seeds but no standard deviations or significance tests, so differences below one point cannot be distinguished from noise. Please report mean plus/minus standard deviation over the three seeds for all main tables, and clarify whether the comparison target is KDPL specifically or all listed methods.
  3. [Eqs. (14)-(16)] Equation (16) as written is degenerate for pairs assigned to the same pseudo-label: in Eq. (14), D^(2)_{c,c'} is the zero vector when c=c', so the normalized direction Dhat^(2) is undefined and the hinge term contributes a constant delta. If the indicator 1[y_i,y_j] is intended to select only pairs of different classes, this must be stated explicitly; otherwise the second-order PCD loss is not well-defined. This issue bears directly on the claimed benefit of the PCD module in Table 6 and Section 5.5.
  4. [Secs. 3.4-3.5 / Sec. 4.1.2] The final objective in Eq. (17) depends on five hyperparameters: alpha in Eq. (13), delta in Eq. (16), and lambda_neg, lambda_diff1, and lambda_diff2. None of these values is given in the experimental setup or in the ablations. Without them the method is not reproducible and the ablation results in Section 5.5 cannot be interpreted quantitatively. Please report the chosen values and, if they vary across the 11 datasets, list them per dataset.
minor comments (6)
  1. [Sec. 4.2.1] The text refers to 'Regional Image Logits (RIL)' while Section 3.2 defines 'Region Information Lookup (RIL)'; please unify the terminology.
  2. [Fig. 8] The Figure 8 caption says results are averaged over seeds 1, 2, and 3, while Section 4.1.2 states that three runs use seeds 0, 1, and 2; please correct the inconsistency.
  3. [Contributions list] The second contribution bullet contains a typo: 'Prompot' should be 'Prompt'.
  4. [Sec. 5.2] The text cites 'CoOpOp [60]' but the reference is CoCoOp; please correct the name.
  5. [Fig. 9] The caption says 'using CoCo and PromptSRC base models'; this should presumably read 'CoOp and PromptSRC'.
  6. [Eq. (10)] The negative loss mixes notation for 'uniform predictions' and the expression 1/|Y| log((1/|Y|)/pbar_i(c)); please clarify that this is the KL divergence from the uniform distribution to the negative-prompt distribution.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the supervision and losses are empirical design choices, and the offline-cost question is a timing-accounting issue, not a circular derivation.

full rationale

Walking the derivation chain, Section 3.2 pre-computes teacher soft labels into the RIL table, and Sections 3.3-3.4 define L_pos, L_neg, L_diff^(1), and L_diff^(2) as training objectives that consume those cached labels and region information weights. Equation 17 is a weighted sum of those losses; none of these equations makes a benchmark quantity equal to an input by construction. Accuracy results are measured on held-out base/new splits and external benchmarks, and hyperparameters such as M=500 and K=20 are tuned in the usual way, which raises ordinary overfitting risk but not circularity. The 2.2x speed claim is based on per-epoch training time in Figure 8 rather than full end-to-end wall clock including one-time RIL construction; this is a potentially serious measurement and reporting concern, but it is not a case of a predicted quantity reducing to a fitted value or to a self-citation. No uniqueness theorem is invoked, and no load-bearing argument rests on the authors' own prior work: the label-sparsification strategies are attributed to external works FKD and FerKD, and the authors' other self-citations are contextual references to their prior methods. The central contribution is an empirical method evaluated against external baselines, so the honest finding is no significant circularity.

Assumptions & free parameters 7 free parameters · 4 assumptions · 0 invented entities

The central claim rests on several user-set hyperparameters and domain assumptions. Five loss weights are never reported, K is tuned on the evaluation benchmark, and M is set without ablation. The entropy-weighting and difference-space assumptions are ad hoc modeling choices.

free parameters (7)
  • alpha
    Weight in Eq. 13 that balances intra-class diversity for high-information regions; value is not reported in the paper.
  • delta
    Margin in the second-order hinge loss Eq. 16; value is not reported.
  • lambda_neg
    Weight for the negative prompt loss L_neg in Eq. 17; value is not reported.
  • lambda_diff1
    Weight for the first-order difference loss in Eq. 17; value is not reported.
  • lambda_diff2
    Weight for the second-order difference loss in Eq. 17; value is not reported.
  • K = 20
    Top-K sparsification parameter selected as optimal after sweeping K=5 to 25 in Figure 9.
  • M = 500
    Number of random crops per image; set in Section 4.1.1 with no reported ablation, despite text promising one in Section 5.
assumptions (4)
  • domain assumption A larger frozen teacher CLIP (ViT-L/14) with the manual prompt 'A photo of a [CLASS]' provides soft supervision that improves student zero-shot generalization.
    Invoked in Section 3.2 during shared supervision generation; the entire distillation benefit depends on this.
  • domain assumption The entropy of the teacher's soft-label distribution reliably measures the semantic information content of an image region.
    Used in Eq. 6 to compute the information weight w_i, which drives the positive/negative prompt split in Section 3.3.
  • domain assumption Random crops, including low-information regions, contain transferable signal that teaches the model to reject uncertain inputs.
    Motivates RADP in Section 3.3 and the claim that low-info regions share latent representations across datasets in Section 4.2.2.
  • ad hoc to paper First-order and second-order differences of positive and negative text features encode useful intra-class and inter-class structure.
    This is the core modeling bet of the PCD module in Section 3.4; no evidence beyond the reported experiments supports it.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FDBPL: Faster Distillation-Based Prompt Learning for Region-Aware Vision-Language Models Adaptation." pith.science (2026). https://pith.science/paper/SQUZU7FM

@misc{pith2026250518053,
  author       = {Pith},
  title        = {Pith review of: FDBPL: Faster Distillation-Based Prompt Learning for Region-Aware Vision-Language Models Adaptation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SQUZU7FM}},
  note         = {Machine review of arXiv:2505.18053}
}
abstract

Prompt learning as a parameter-efficient method that has been widely adopted to adapt Vision-Language Models (VLMs) to downstream tasks. While hard-prompt design requires domain expertise and iterative optimization, soft-prompt methods rely heavily on task-specific hard labels, limiting their generalization to unseen categories. Recent popular distillation-based prompt learning methods improve generalization by exploiting larger teacher VLMs and unsupervised knowledge transfer, yet their repetitive teacher model online inference sacrifices the inherent training efficiency advantage of prompt learning. In this paper, we propose {{\large {\textbf{F}}}}aster {{\large {\textbf{D}}}}istillation-{{\large {\textbf{B}}}}ased {{\large {\textbf{P}}}}rompt {{\large {\textbf{L}}}}earning (\textbf{FDBPL}), which addresses these issues by sharing soft supervision contexts across multiple training stages and implementing accelerated I/O. Furthermore, FDBPL introduces a region-aware prompt learning paradigm with dual positive-negative prompt spaces to fully exploit randomly cropped regions that containing multi-level information. We propose a positive-negative space mutual learning mechanism based on similarity-difference learning, enabling student CLIP models to recognize correct semantics while learning to reject weakly related concepts, thereby improving zero-shot performance. Unlike existing distillation-based prompt learning methods that sacrifice parameter efficiency for generalization, FDBPL maintains dual advantages of parameter efficiency and strong downstream generalization. Comprehensive evaluations across 11 datasets demonstrate superior performance in base-to-new generalization, cross-dataset transfer, and robustness tests, achieving $2.2\times$ faster training speed.

Figures

Figures reproduced from arXiv: 2505.18053 by the authors.

Figure 1
Figure 1. Training Efficiency Advantage. The results are measured in min￾utes, and our method demonstrates significant improvements across all 11 datasets, as well as in average training time. arXiv:2505.18053v1 [cs.CV] 23 May 2025 [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. From PL to FDBPL (Ours). Prompt learning (PL) adapts CLIP to downstream tasks via learnable parameters. We compare three methods: (a) Native PL employs dataset hard labels, offering fast training but suffering from overfitting on seen classes, which degrades zero-shot performance on unseen classes. (b) Distillation-Based PL uses a teacher CLIP network to transfer generalization knowledge without specific labels, oft… view at source ↗
Figure 3
Figure 3. Relationships between Components within FDBPL. The training paradigm based on random region images inevitably include regions with in￾sufficient information content (blue areas) compared to well-defined regions of interest (ROI) under sharp distributions (red areas). To address this chal￾lenge, we implement RADP (Region-Adaptive Dual Prompt) learning, which uses positive prompts for high-information regions and nega… view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: FDBPL Framework. (a) To mitigate native prompt learning’s strong dependency on hard-labeled downstream dataset, we introduce a larger teacher CLIP network that transfers generalized knowledge through unlabeled regional images. For efficient knowledge distillation, we p…
Figure 5
Figure 5. Figure 5: Label Sparsity Strategy. A large-capacity teacher network, such as CLIP trained on ImageNet-1K, produces a logit output with 1000 categories (a). Storing these complete logits as "Soft Label" in the Region Information Lookup (RIL) Table (b) incurs significant storage o…
Figure 6
Figure 6. Figure 6: Region-Aware Dual-Prompt (RADP) Learning. Random sampling of regional images from the Region Information Lookup (RIL) Table inevitably yields regions with varying information content, where larger weights (w) indi￾cate richer information. To address this, we introduce …
Figure 7
Figure 7. Figure 7: Prompt-Cascaded Difference (PCD) Learning. (a) When calculat￾ing the first-order intra-class similarity S (1) in the first-order difference space, we expect regions with high information content (larger values of w) to gener￾ate more diverse intra-class distributions b…
Figure 8
Figure 8. Figure 8: Training Efficiency Gain. Following the base-to-new protocol, this figure quantifies both the training time consumption of our proposed FDBPL and its efficiency improvement over SOTA methods. All experiments are executed on a single NVIDIA 3090 GPU, with results averag…
Figure 9
Figure 9. Figure 9: presents the results of this analysis, illustrating key [PITH_FULL_IMAGE:figures/full_fig_p009_9.png]
Figure 10
Figure 10. Figure 10: Ablation analysis of the FDBPL component. This analysis evaluates the effectiveness of the Similarity-Training (Lpos and Lneg) and Difference￾Training (L (1) di f f and L (2) di f f ) strategies employed within FDBPL. Furthermore, Memory Consumption relative to the ba…
Figure 11
Figure 11. Figure 11: Attention Region Visualization. (A)illustrates visual image inputs from ImageNet-1K and their corresponding text labels; (B) demonstrates the attention regions focused by comparative methods. CLIP employs a hard prompt template "A photo of a [class]" to guide semantic…
Figure 12
Figure 12. Figure 12: Effect of PCD on Wrong Case Analysis. We present the performance of FDBPL with PCD on challenging test cases. Results demonstrate that the inherent intra-class and inter-class latent relationships captured by PCD benefit zero-shot performance in complex scenarios. For…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

64 extracted references · 54 canonical work pages

  1. [1]

    Radford, J

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al., Learning transferable visual models from natural language supervision, in: International conference on ma- chine learning, PmLR, 2021, pp. 8748–8763

    work page 2021

  2. [2]

    Y . Ge, J. Xu, B. N. Zhao, N. Joshi, L. Itti, V . Vi- neet, Dall-e for detection: Language-driven composi- tional image synthesis for object detection, arXiv preprint arXiv:2206.09592 (2022)

    arXiv 2022

  3. [3]

    Rombach, A

    R. Rombach, A. Blattmann, D. Lorenz, P. Esser, B. Om- mer, High-resolution image synthesis with latent diffusion models (2021).arXiv:2112.10752

    arXiv 2021

  4. [4]

    Zhang, C

    Z. Zhang, C. Wang, R. Xu, W. Xu, S. Xu, L. Guo, J. Zhang, X. Teng, W. Xu, Mim-hd: Making smaller masked autoencoder better with efficient distillation, in: European Conference on Artificial Intelligence, 2024. URLhttps://api.semanticscholar.org/ CorpusID:273589355

    work page 2024

  5. [5]

    Y . Sun, X. Wang, J. Huang, S. Chen, M. Hou, Reusable generator data-free knowledge distillation with hard loss simulation for image classification, Expert Systems with Applications 265 (2025) 126025

    work page 2025

  6. [6]

    D. Ma, K. Zhang, Q. Cao, J. Li, X. Gao, Coordinate at- tention guided dual-teacher adaptive knowledge distilla- tion for image classification, Expert Systems with Appli- cations 250 (2024) 123892. 15

    work page 2024

  7. [7]

    Z. Lu, Y . Lu, Disentangling the intrinsic feature from the related feature in image classification using knowledge distillation and object replacement, Expert Systems with Applications 246 (2024) 123193

    work page 2024

  8. [8]

    X. Yu, L. Teng, D. Zhang, J. Zheng, H. Chen, Attention correction feature and boundary constraint knowledge dis- tillation for efficient 3d medical image segmentation, Ex- pert Systems with Applications 262 (2025) 125670

    work page 2025

Show all 64 references
  1. [9]

    Y . Su, S. Chen, Y .-G. Wang, Balanced residual distilla- tion learning for 3d point cloud class-incremental seman- tic segmentation, Expert Systems with Applications 269 (2025) 126399

    2025

  2. [10]

    K. Tan, J. Tang, Z. Zhao, C. Wang, H. Miao, X. Zhang, X. Chen, Efficient and lightweight layer-wise in-situ de- fect detection in laser powder bed fusion via knowledge distillation and structural re-parameterization, Expert Sys- tems with Applications 255 (2024) 124628

    2024

  3. [11]

    Y . Chen, M. Lin, Z. He, K. Polat, A. Alhudhaif, F. Alenezi, Consistency-and dependence-guided knowl- edge distillation for object detection in remote sensing images, Expert Systems with Applications 229 (2023) 120519

    2023

  4. [12]

    Y . Liu, Y . Zhang, R. Lan, C. Cheng, Z. Wu, Awardistill: Adaptive and robust 3d object detection in adverse condi- tions through knowledge distillation, Expert Systems with Applications 266 (2025) 126032

    2025

  5. [13]

    T. Xiao, R. Pang, H. Liu, C. Yang, A. Li, C. Niu, Z. Ruan, L. Xu, Y . Ge, Domain adaptation and knowledge distil- lation for lightweight pavement crack detection, Expert Systems with Applications 263 (2025) 125734

    2025

  6. [14]

    X. Guo, W. Zhou, T. Liu, Multilevel attention imitation knowledge distillation for rgb-thermal transmission line detection, Expert Systems with Applications 260 (2025) 125406

    2025

  7. [15]

    R. Xu, C. Wang, D. Zhang, M. Zhang, S. Xu, W. Meng, X. Zhang, Deffusion: Deformable multimodal represen- tation fusion for 3d semantic segmentation, 2024 IEEE International Conference on Robotics and Automation (ICRA) (2024) 7732–7739. URLhttps://api.semanticscholar.org/ Corpu...

    2024

  8. [16]

    C. Wang, R. Xu, S. Xu, W. Meng, R. Wang, X. Zhang, Exploring intrinsic discrimination and consistency for weakly supervised object localization, IEEE Transactions on Image Processing 33 (2024) 1045–1058. URLhttps://api.semanticscholar.org/ CorpusID:267256165

    2024

  9. [17]

    P. Ren, M. Li, Z. Luo, X. Song, Z. Chen, W. Li- ufu, Y . Yang, H. Zheng, R. Xu, Z. Huang, et al., In- finiteworld: A unified scalable simulation framework for general visual-language robot interaction, arXiv preprint arXiv:2412.05789 (2024)

    2024 arXiv

  10. [18]

    R. Xu, J. Zhang, M. Guo, Y . Wen, H. Yang, M. Lin, J. Huang, Z. Li, K. Zhang, L. Wang, et al., A0: An affordance-aware hierarchical model for general robotic manipulation, arXiv preprint arXiv:2504.12636 (2025)

    2025

  11. [19]

    Liang, M

    X. Liang, M. Lin, W. Ruan, R. Xu, Y . Liu, J. Chen, B. Lin, Y . Zhuang, X. Liang, Structured preference opti- mization for vision-language long-horizon task planning, arXiv preprint arXiv:2502.20742 (2025)

    2025

  12. [20]

    Q. Xu, J. Li, Y . Tian, L. Zhou, F. Zhang, R. Huang, Diff- zsvqa: Zero-shot visual question answering with frozen large language models using diffusion model, Expert Sys- tems with Applications (2025) 126951

    2025

  13. [21]

    D. Han, J. Shi, J. Zhao, H. Wu, Y . Zhou, L.-H. Li, M. K. Khan, K.-C. Li, Lrcn: Layer-residual co-attention net- works for visual question answering, Expert Systems with Applications 263 (2025) 125658

    2025

  14. [22]

    Zhang, X

    J. Zhang, X. Ma, S. Guo, P. Li, W. Xu, X. Tang, Z. Hong, Amend to alignment: decoupled prompt tuning for miti- gating spurious correlation in vision-language models, in: Forty-first International Conference on Machine Learn- ing, 2024

    2024

  15. [23]

    B. Zhu, Y . Niu, Y . Han, Y . Wu, H. Zhang, Prompt-aligned gradient for prompt tuning, International Conference on Computer Vision, 2023

    2023

  16. [24]

    C. X. Hantao Yao, Rui Zhang, Visual-language prompt tuning with knowledge-guided context optimization, in: The IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, 2023

    2023

  17. [25]

    Mistretta, A

    M. Mistretta, A. Baldrati, M. Bertini, A. D. Bagdanov, Im- proving zero-shot generalization of learned prompts via unsupervised knowledge distillation, in: European Con- ference on Computer Vision, Springer, 2024, pp. 459– 477

    2024

  18. [26]

    Z. Li, X. Li, X. Fu, X. Zhang, W. Wang, S. Chen, J. Yang, Promptkd: Unsupervised prompt distillation for vision- language models, in: Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, 2024, pp. 26617–26626

    2024

  19. [27]

    J. Gao, J. Ruan, S. Xiang, Z. Yu, K. Ji, M. Xie, T. Liu, Y . Fu, Lamm: Label alignment for multi-modal prompt learning, in: Proceedings of the AAAI Conference on Ar- tificial Intelligence, V ol. 38, 2024, pp. 1815–1823

    2024

  20. [28]

    J. Ruan, J. Gao, M. Xie, S. Xiang, Z. Yu, T. Liu, Y . Fu, X. Qu, Gist: Improving parameter efficient fine-tuning via knowledge interaction, in: Proceedings of the 32nd ACM International Conference on Multimedia, 2024, pp. 8835– 8844. 16

    2024

  21. [29]

    Z. Shen, E. Xing, A fast knowledge distillation framework for visual recognition, in: European conference on com- puter vision, Springer, 2022, pp. 673–690

    2022

  22. [30]

    Shen, Ferkd: Surgical label adaptation for efficient dis- tillation, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp

    Z. Shen, Ferkd: Surgical label adaptation for efficient dis- tillation, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 1666–1675

    2023

  23. [31]

    Zhang, C

    Z. Zhang, C. Wang, R. Xu, W. Xu, S. Xu, Y . Zhang, L. Guo, Cae-dfkd: Bridging the transferability gap in data-free knowledge distillation, 2025. URLhttps://api.semanticscholar.org/ CorpusID:278207768

    2025

  24. [32]

    Zhang, R

    Z. Zhang, R. Xu, C. Wang, W. Xu, S. Chen, S. Xu, G. Xu, L. Guo, Dfmc:feature-driven data-free knowledge distil- lation, IEEE Transactions on Circuits and Systems for Video Technology (2025) 1–1doi:10.1109/TCSVT. 2025.3565616

    2025

  25. [33]

    C. Wang, R. Xu, S. Xu, W. Meng, X. Zhang, Cndesc: Cross normalization for local descriptors learning, IEEE Transactions on Multimedia 25 (2023) 3989–4001. URLhttps://api.semanticscholar.org/ CorpusID:248330052

    2023

  26. [34]

    C. Wang, S. Chen, Y . Song, R. Xu, Z. Zhang, J. Zhang, H. Yang, Y . Zhang, K. Fu, S. Du, Z. Xu, L. Gao, L. Guo, S. Xu, Focus on local: Finding reliable discriminative re- gions for visual place recognition, in: AAAI Conference on Artificial Intelligence, 2025. URLhttps://api.s...

    2025

  27. [35]

    W. Xu, C. Wang, R. Xu, S. Xu, W. Meng, M. Zhang, X. Zhang, Token masking transformer for weakly supervised object localization, IEEE Transactions on Multimedia 27 (2025) 2059–2069. URLhttps://api.semanticscholar.org/ CorpusID:275050355

    2025

  28. [36]

    Sameni, K

    S. Sameni, K. Kafle, H. Tan, S. Jenni, Building vision- language models on solid foundations with masked dis- tillation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 14216–14226

    2024

  29. [37]

    P. K. A. Vasu, H. Pouransari, F. Faghri, R. Vemulapalli, O. Tuzel, Mobileclip: Fast image-text models through multi-modal reinforced training, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 15963–15974

    2024

  30. [38]

    K. Wu, H. Peng, Z. Zhou, B. Xiao, M. Liu, L. Yuan, H. Xuan, M. Valenzuela, X. S. Chen, X. Wang, et al., Tinyclip: Clip distillation via affinity mimicking and weight inheritance, in: Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision, 2023, pp. 21970–21980

    2023

  31. [39]

    C. Yang, Z. An, L. Huang, J. Bi, X. Yu, H. Yang, B. Diao, Y . Xu, Clip-kd: An empirical study of clip model dis- tillation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 15952–15962

    2024

  32. [40]

    K. Yang, T. Gu, X. An, H. Jiang, X. Dai, Z. Feng, W. Cai, J. Deng, Clip-cid: Efficient clip distillation via cluster- instance discrimination, arXiv preprint arXiv:2408.09441 (2024)

    2024 arXiv

  33. [41]

    K. Zhou, J. Yang, C. C. Loy, Z. Liu, Learning to prompt for vision-language models, International Journal of Com- puter Vision 130 (9) (2022) 2337–2348

    2022

  34. [42]

    K. Zhou, J. Yang, C. C. Loy, Z. Liu, Conditional prompt learning for vision-language models, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 16816–16825

    2022

  35. [43]

    M. U. Khattak, H. Rasheed, M. Maaz, S. Khan, F. S. Khan, Maple: Multi-modal prompt learning, in: Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, 2023, pp. 19113–19122

    2023

  36. [44]

    M. U. Khattak, S. T. Wasim, M. Naseer, S. Khan, M.-H. Yang, F. S. Khan, Self-regulating prompts: Foundational model adaptation without forgetting, in: Proceedings of the IEEE/CVF international conference on computer vi- sion, 2023, pp. 15190–15200

    2023

  37. [45]

    Huang, J

    T. Huang, J. Chu, F. Wei, Unsupervised prompt learning for vision-language models, arXiv preprint arXiv:2204.03649 (2022)

    2022 arXiv

  38. [46]

    X. Tian, S. Zou, Z. Yang, J. Zhang, Argue: Attribute- guided prompt tuning for vision-language models, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 28578–28587

    2024

  39. [47]

    J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, L. Fei- Fei, Imagenet: A large-scale hierarchical image database, in: 2009 IEEE conference on computer vision and pattern recognition, Ieee, 2009, pp. 248–255

    2009

  40. [48]

    Fei-Fei, R

    L. Fei-Fei, R. Fergus, P. Perona, Learning generative vi- sual models from few training examples: An incremen- tal bayesian approach tested on 101 object categories, in: 2004 conference on computer vision and pattern recogni- tion workshop, IEEE, 2004, pp. 178–178

    2004

  41. [49]

    O. M. Parkhi, A. Vedaldi, A. Zisserman, C. Jawahar, Cats and dogs, in: 2012 IEEE conference on computer vision and pattern recognition, IEEE, 2012, pp. 3498–3505

    2012

  42. [50]

    Krause, M

    J. Krause, M. Stark, J. Deng, L. Fei-Fei, 3d object repre- sentations for fine-grained categorization, in: Proceedings of the IEEE international conference on computer vision workshops, 2013, pp. 554–561. 17

    2013

  43. [51]

    Nilsback, A

    M.-E. Nilsback, A. Zisserman, Automated flower classi- fication over a large number of classes, in: 2008 Sixth Indian conference on computer vision, graphics & image processing, IEEE, 2008, pp. 722–729

    2008

  44. [52]

    Bossard, M

    L. Bossard, M. Guillaumin, L. Van Gool, Food-101– mining discriminative components with random forests, in: Computer vision–ECCV 2014: 13th European con- ference, zurich, Switzerland, September 6-12, 2014, pro- ceedings, part VI 13, Springer, 2014, pp. 446–461

    2014

  45. [53]

    S. Maji, E. Rahtu, J. Kannala, M. Blaschko, A. Vedaldi, Fine-grained visual classification of aircraft, arXiv preprint arXiv:1306.5151 (2013)

    2013 arXiv

  46. [54]

    J. Xiao, J. Hays, K. A. Ehinger, A. Oliva, A. Torralba, Sun database: Large-scale scene recognition from abbey to zoo, in: 2010 IEEE computer society conference on computer vision and pattern recognition, IEEE, 2010, pp. 3485–3492

    2010

  47. [55]

    Soomro, A

    K. Soomro, A. R. Zamir, M. Shah, Ucf101: A dataset of 101 human actions classes from videos in the wild, arXiv preprint arXiv:1212.0402 (2012)

    2012 arXiv

  48. [56]

    Cimpoi, S

    M. Cimpoi, S. Maji, I. Kokkinos, S. Mohamed, A. Vedaldi, Describing textures in the wild, in: Proceed- ings of the IEEE conference on computer vision and pat- tern recognition, 2014, pp. 3606–3613

    2014

  49. [57]

    Helber, B

    P. Helber, B. Bischke, A. Dengel, D. Borth, Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sens- ing 12 (7) (2019) 2217–2226

    2019

  50. [58]

    Miyai, Q

    A. Miyai, Q. Yu, G. Irie, K. Aizawa, Locoop: Few- shot out-of-distribution detection via prompt learning, Advances in Neural Information Processing Systems 36 (2023) 76298–76310

    2023

  51. [59]

    M. Jia, L. Tang, B.-C. Chen, C. Cardie, S. Belongie, B. Hariharan, S.-N. Lim, Visual prompt tuning, in: Euro- pean conference on computer vision, Springer, 2022, pp. 709–727

    2022

  52. [60]

    K. Zhou, J. Yang, C. C. Loy, Z. Liu, Conditional prompt learning for vision-language models, in: IEEE/CVF Con- ference on Computer Vision and Pattern Recognition (CVPR), 2022

    2022

  53. [61]

    Recht, R

    B. Recht, R. Roelofs, L. Schmidt, V . Shankar, Do ima- genet classifiers generalize to imagenet?, in: International conference on machine learning, PMLR, 2019, pp. 5389– 5400

    2019

  54. [62]

    H. Wang, S. Ge, Z. Lipton, E. P. Xing, Learning ro- bust global representations by penalizing local predictive power, Advances in neural information processing sys- tems 32 (2019)

    2019

  55. [63]

    Hendrycks, K

    D. Hendrycks, K. Zhao, S. Basart, J. Steinhardt, D. Song, Natural adversarial examples, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 15262–15271

    2021

  56. [64]

    Hendrycks, S

    D. Hendrycks, S. Basart, N. Mu, S. Kadavath, F. Wang, E. Dorundo, R. Desai, T. Zhu, S. Parajuli, M. Guo, et al., The many faces of robustness: A critical analysis of out-of-distribution generalization, in: Proceedings of the IEEE/CVF international conference on computer vision...

    2021

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.