Pith. sign in

REVIEW 4 major objections 10 minor 52 references

Structured-Condensed Prompt Tuning in Vision-Language Models for Fine-grained Image Recognition

T0 review · 4 major / 10 minor · reviewed 2026-07-08 · glm-5.2

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Encoding inter-class relationships into CLIP prompts boosts fine-grained recognition

desk verdict SCPT combines signed random projection (LSH) for inter-class semantic encoding with SVD-based denoising of the KgCoOp regularization target. Consistent ~1% gains over TCP across 14 fine-grained benchmarks, but the ablation has a load-bearing gap: no random-binary-code control to isolate whether semantic structure or extra token capacity drives the gains. read the letter →

arxiv 2607.06185 v1 pith:XKQDBCID submitted 2026-07-07 cs.CV

Xinda Liu , Qinyu Zhang , Weiqing Min , Guohua Geng , Shuqiang Jiang This is my paper · ORCID
classification cs.CV
keywords semanticfine-grainedpromptrecognitiontuningabilitycapturecomponents
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that existing prompt-tuning methods for vision-language models like CLIP treat each class label as an isolated token, ignoring the semantic relationships between categories that matter most when categories are visually similar. The authors propose Structured-Condensed Prompt Tuning (SCPT), which injects two things into the standard learnable-prompt framework: (1) Semantic Relation Encoding (SRE), which computes pairwise cosine similarities between CLIP text embeddings of all class names, compresses the resulting N×N similarity matrix into compact binary codes via signed random projection, and prepends these codes to the learnable prompt vectors so each class prompt carries information about where it sits relative to every other class; and (2) Semantic Condensation loss (ScLoss), which applies truncated singular value decomposition to the handcrafted-prompt embeddings used as a regularization target, removing small singular values likely to represent noise (guided by Marchenko-Pastur theory for pure-noise matrices) before computing the MSE between learnable and denoised reference embeddings. The combination is designed so that SRE expands inter-class discriminability while ScLoss contracts intra-class redundancy. Across 14 fine-grained benchmarks spanning food, animals, vehicles, aircraft, and flowers, SCPT reportedly achieves state-of-the-art results in both few-shot classification and base-to-novel generalization, surpassing the prior best method TCP by roughly 1% on average while using only shallow, text-only prompts comparable in parameter count and inference speed to TCP.

What carries the argument

Semantic Relation Encoding (SRE): signed random projection of the CLIP-derived inter-class cosine similarity matrix into binary codes, prepended to learnable prompt vectors. Semantic Condensation loss (ScLoss): SVD-based truncation of handcrafted-prompt embeddings with adaptive threshold selection via Marchenko-Pastur noise bounds and a probabilistic retention function, used as a denoised MSE regularization target.

What would settle it

Replace the SRE binary codes with random binary vectors of the same dimension. If classification performance remains unchanged, the semantic content of the codes is not the source of the improvement.

Watch

Extended reading notes

Core claim

The central mechanism is the SRE token: a binary code of dimension d = ceil(log2(N) + d_free) bits, produced by projecting the N×N class-similarity matrix (derived from frozen CLIP text embeddings of class names) through a random Gaussian matrix and taking the sign. Each class's SRE token encodes its angular relationships to all other classes in Hamming space, where the expected normalized Hamming distance between two classes' codes equals their angular distance in the original embedding space divided by pi (a classical locality-sensitive-hashing result). These binary tokens are embedded via the standard word-embedding function and concatenated with learnable context vectors before being fed

Load-bearing premise

The SRE module assumes that CLIP's frozen text embeddings of class names (from the template 'a photo of [classname]') encode meaningful taxonomic relationships that survive compression into binary codes and improve classification when prepended to learnable prompts. The Johnson-Lindenstrauss lemma guarantees approximate distance preservation, but not that the preserved distances are the ones that matter for distinguishing fine-grained categories.

Editorial extensions

If this is right

  • If inter-class semantic topology is as beneficial as reported, datasets with larger class counts should see proportionally larger gains from SRE, since richer category sets produce more informative topology — the paper notes this trend in t-SNE visualizations but does not systematically quantify it.
  • The SRE approach could extend to open-vocabulary or hierarchical classification settings where class relationships are explicitly taxonomic, potentially amplifying the structure-preserving benefit.
  • The Marchenko-Pastur-guided singular-value truncation in ScLoss is a general denoising technique for any prompt-regularization scheme that uses frozen reference embeddings, not specific to fine-grained recognition.
  • If shallow text-only prompting with structural encoding matches deep multi-modal methods (as the comparison with PromptSRC suggests), the field may need to reconsider whether prompt depth and modality are the right axes of progress, or whether structural inductive biases are more cost-effective.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not ablate the information content of the binary SRE codes themselves (e.g., by replacing them with random binary vectors of the same dimension). If such a control showed comparable gains, it would suggest the benefit comes from added prompt capacity or regularization dynamics rather than from preserved semantic topology.
  • The signed random projection preserves angular distances in expectation, but classification utility depends on whether the preserved distances are the discriminative ones for fine-grained classes — classes that are semantically close but visually distinct (or vice versa) could produce SRE codes that mislead rather than help.
  • The ScLoss threshold tau=0.3 is fixed across all datasets, but the optimal truncation rank K is computed per-dataset via the probabilistic selection function. The interaction between dataset-specific K and the fixed tau is not fully explored — it is possible that the method's robustness to tau stems from the probabilistic retention function absorbing most of the sensitivity.
  • The comparison with PromptSRC (deep, multi-modal, 76.94% vs. SCPT's 76.70%) is close enough that the ~0.24% gap could fall within seed variance; the paper does not report confidence intervals or multiple-seed averages for this comparison.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 10 minor

Summary. The paper proposes Structured-Condensed Prompt Tuning (SCPT) for fine-grained image recognition with vision-language models. SCPT has two components: (1) Semantic Relation Encoding (SRE), which computes a class similarity matrix from CLIP's frozen text embeddings, compresses it via signed random projection into binary codes, and prepends these as tokens to learnable prompts; and (2) Semantic Condensation loss (ScLoss), which applies SVD-based truncation to handcrafted prompt embeddings before computing an MSE regularizer against learnable prompt embeddings. The method is evaluated on 14 fine-grained benchmarks in few-shot and base-to-novel generalization settings, reporting state-of-the-art results with a shallow, text-only design.

Significance. The paper addresses a well-motivated problem: most prompt tuning methods treat class labels as independent tokens, ignoring inter-class semantic structure. The combination of signed random projection (grounded in the JL lemma and LSH theory, with a concentration bound in Appendix C) and SVD-based denoising (motivated by Marchenko-Pastur) is a reasonable and lightweight approach. The experimental coverage is broad (14 datasets, multiple backbones, efficiency analysis). The method achieves competitive results with a shallow text-only design comparable to deeper multi-modal approaches like PromptSRC. The reproducibility of the random projection component is supported by a seed-sensitivity analysis (Table 4).

major comments (4)
  1. [Table 3, §4.4] The central narrative is that SRE explicitly encodes inter-class semantic topology and this is a key innovation. However, Table 3 shows SRE alone contributes only +0.17% over the TCP baseline (75.80 vs. 75.63), which is within typical run-to-run variance. The combined SRE+ScLoss gain of +1.07% is presented as evidence of synergy, but no control experiment replaces SRE with random binary codes of the same dimension. Without this control, the paper cannot distinguish 'semantic structure helps' from 'extra learnable token positions help.' This is load-bearing because the paper's core claim is that structured semantic modeling is the mechanism. A random-binary-code ablation would directly test whether the semantic content of SRE codes matters beyond providing additional prompt capacity.
  2. [§3.2.3, Eq. (15)] The Marchenko-Pastur motivation for SVD truncation is applied heuristically. The threshold τ=0.3 is fixed across all 14 datasets without rigorous justification. While Figure 7 shows low sensitivity to τ on four datasets, the theoretical connection between the MP distribution and the fixed threshold τ=0.3 is not established. Specifically, Eq. (15) defines a probabilistic selection function P(σ_i), but the relationship between τ and the MP upper bound σ_max (which depends on N, M, and noise variance σ²) is never made explicit. If τ is meant to approximate σ_max, the paper should show this; if not, the MP motivation is decorative. This weakens the theoretical grounding of the ScLoss component.
  3. [§3.2.1, Eq. (6)-(7)] The SRE binary codes P are passed through the word embedding function e(·) to generate tokens R_i = e(P_i), where P_i ∈ {0,1}^d. The paper does not discuss what e(·) maps binary values 0 and 1 to in CLIP's vocabulary. If 0 maps to a padding or special token and 1 maps to a regular token, the semantic content of the binary codes may be lost or distorted at the embedding stage. The JL lemma (Appendix C) guarantees approximate angular distance preservation in Hamming space, but this says nothing about whether CLIP's text encoder can meaningfully process these binary codes as input tokens. The paper should clarify the embedding mechanism and discuss whether the binary codes carry usable signal through e(·).
  4. [Table 2, §4.3] In the base-to-novel generalization setting, the SRE for novel classes is computed from textual semantic representations of candidate class names using the same random projection function as in training (§4.3). However, it is unclear whether the random projection matrix W is fixed at training time and reused at test time, or re-generated. If W is regenerated, the binary codes for base classes would change between training and inference, breaking consistency. The paper should explicitly state whether W is fixed or re-randomized for novel-class inference, as this affects the validity of the generalization results.
minor comments (10)
  1. [Abstract] The abstract states 'achieves state-of-the-art performance' but Table 7 shows PromptSRC achieves 76.94% vs. SCPT's 76.70% on the same 14-dataset average. The claim should be qualified.
  2. [Table 1] The table caption says '14 fine-grained image datasets' but lists 14 datasets including 'Dog Breed' and 'Stanford Dogs' which are both dog datasets. Clarify whether these overlap.
  3. [§3.2.2, Eq. (11)] The variable N_c (number of seen classes) is mentioned in the text after Eq. (11) but the equation uses N. Consistency is needed.
  4. [§4.1] The implementation details state 'threshold τ is 0.3' but do not specify the value of d_free used in the main experiments. Figure 4 suggests d_free=3 is best, but this should be stated explicitly.
  5. [Table 2(d), Oxford Pets] SCPT underperforms ProText on Oxford Pets in the base-to-novel setting (HM 96.29 vs. 96.45). The paper does not discuss this exception, while it does discuss Oxford Pets in the few-shot setting (§4.2).
  6. [Figure 3] The figure caption refers to 'SVD Dropout' but the method is described as 'SVD-based filtering' or 'truncation' in the text. Terminology should be consistent.
  7. [§4.4, Table 3] The ablation uses 'TCP (Reg.)' as baseline but it is described as 'CoOp-style prompt tuning with semantic regularization.' Clarify whether this is exactly the TCP method or a variant.
  8. [Appendix C] Lemma 1 and Theorem 1 are standard LSH results. The paper should cite the original source (e.g., Charikar, 2002) rather than only referencing 'classical result in LSH' without citation.
  9. [Table 5] The 'Ratio (%)' column header is ambiguous — ratio of what to what? Presumably learnable parameters to total model parameters, but this should be stated.
  10. [§4.5, Table 7] The comparison with PromptSRC uses 14 datasets for SCPT but the caption says '15 datasets.' This inconsistency should be corrected.

Simulated Author's Rebuttal

4 responses · 0 unresolved

We thank the referee for a careful and constructive review. The four major comments raise substantive points about experimental controls, theoretical grounding, embedding mechanics, and implementation details. We address each below and commit to revisions where the referee's points are well-taken.

read point-by-point responses

  1. Referee: [Table 3, §4.4] SRE alone contributes only +0.17% over TCP baseline, within run-to-run variance. No control experiment replaces SRE with random binary codes of the same dimension. Without this control, the paper cannot distinguish 'semantic structure helps' from 'extra learnable token positions help.'

    Authors: The referee raises a valid and important concern. We agree that the random-binary-code ablation is a necessary control to isolate the contribution of semantic structure from the effect of additional learnable token positions. We will conduct this experiment by replacing the SRE codes with random binary vectors of the same dimension (drawn uniformly from {0,1}^d with d = ceil(log2(N) + d_free)), keeping all other components fixed. This will directly test whether the semantic content of SRE codes matters beyond providing additional prompt capacity. We will report the results in the revised Table 3. We acknowledge that the +0.17% gain from SRE alone is modest and within typical variance, and the random-code control will allow us to make honest claims about whether SRE's semantic structure provides a genuine benefit or whether the gains primarily come from the synergy with ScLoss. If the random-code baseline matches or exceeds SRE, we will revise our claims accordingly. revision: yes

  2. Referee: [§3.2.3, Eq. (15)] The Marchenko-Pastur motivation for SVD truncation is applied heuristically. The threshold τ=0.3 is fixed across all 14 datasets without rigorous justification. The relationship between τ and the MP upper bound σ_max is never made explicit. If τ is meant to approximate σ_max, the paper should show this; if not, the MP motivation is decorative.

    Authors: The referee is correct that the connection between the fixed threshold τ=0.3 and the MP upper bound σ_max is not made explicit in the current manuscript. We will address this in the revision. Specifically, we will: (1) compute the MP upper bound σ_max for representative datasets using the estimated noise variance and matrix dimensions, and show the relationship between σ_max and the empirically optimal truncation point; (2) clarify that τ serves as a practical approximation to σ_max, and discuss why a single fixed value works reasonably across datasets despite varying N and M (Figure 7 shows sensitivity below 0.3% across the tested range). We concede that the current presentation does not rigorously establish the τ–σ_max relationship, and the MP motivation is presently more of a guiding intuition than a derived result. We will revise the text to be transparent about this: the MP distribution motivates the existence of a noise floor, but the specific threshold is empirically validated rather than theoretically derived. If the reviewer feels the MP framing should be substantially de-emphasized, we are open to that as well. revision: partial

  3. Referee: [§3.2.1, Eq. (6)-(7)] The paper does not discuss what e(·) maps binary values 0 and 1 to in CLIP's vocabulary. If 0 maps to a padding or special token and 1 maps to a regular token, the semantic content of the binary codes may be lost or distorted at the embedding stage. The JL lemma guarantees angular distance preservation in Hamming space, but says nothing about whether CLIP's text encoder can meaningfully process these binary codes as input tokens.

    Authors: This is a fair point that we failed to address in the manuscript. We will clarify the embedding mechanism in the revision. In our implementation, the binary codes P_i are not mapped to specific vocabulary tokens. Instead, following the standard practice in CoOp-style methods where learnable context vectors are continuous embeddings outside the discrete vocabulary space, the binary codes are treated as indices into CLIP's word embedding lookup table. Specifically, the values 0 and 1 are used to index into the embedding matrix, and the resulting embedding vectors are then treated as continuous inputs to the text encoder, analogous to how soft prompt vectors operate. The key distinction from discrete token processing is that these embeddings are not constrained to correspond to meaningful natural language tokens; rather, they serve as structured initialization points that the text encoder processes through its standard transformer layers. We agree that the JL lemma guarantee applies to the Hamming space representation and does not directly extend to the text encoder's processing of these embeddings. We will add a discussion of this limitation and note that the empirical results (including the seed-sensitivity analysis in Table 4) demonstrate that the signal does carry through e(·) in practice, even though the theoretical guarantee does not cover this stage. We will also add the random-binary-code ablation (from Comment 1) as further evidence. revision: yes

  4. Referee: [Table 2, §4.3] It is unclear whether the random projection matrix W is fixed at training time and reused at test time, or re-generated. If W is regenerated, the binary codes for base classes would change between training and inference, breaking consistency. The paper should explicitly state whether W is fixed or re-randomized for novel-class inference.

    Authors: We thank the referee for catching this ambiguity. To clarify: the random projection matrix W is fixed at training time and reused at test time for novel-class inference. W is generated once, stored, and applied consistently to compute SRE codes for both base and novel classes. This is essential for consistency, as the referee correctly notes. We will explicitly state in the revised §4.3 that W is fixed throughout training and inference, and we will also clarify this in the algorithm description (Algorithm 1) and in §3.2.1 where W is introduced. revision: yes

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: SRE and ScLoss are independently constructed from frozen CLIP embeddings and externally evaluated on 14 benchmarks

full rationale

The paper's two main components, SRE and ScLoss, are constructed from frozen CLIP text embeddings of class names and are not defined in terms of the paper's own outputs. SRE computes a similarity matrix S from frozen embeddings (Eq. 5), applies signed random projection (Eq. 6), and prepends the result to learnable prompts (Eq. 7). ScLoss applies SVD denoising to handcrafted prompt embeddings X and regularizes learnable embeddings Y against the denoised version (Eqs. 8-11). Neither component is defined in terms of the prediction it produces. The theoretical justification (Appendix C) invokes the JL lemma and LSH random hyperplane results, which are standard external mathematical facts, not self-citations. The experimental evaluation on 14 benchmarks with comparisons to 7 external methods provides independent falsification. The self-referential information flow (CLIP embeddings used to structure prompts fed back into CLIP) is a design choice, not a circularity: the similarity matrix is computed from frozen, pre-trained embeddings and is not a fitted parameter, and the learnable context vectors are optimized against external image data via contrastive loss. The paper does not fit a parameter to a subset of data and then 'predict' a closely related quantity. The ablation in Table 3 isolates SRE and ScLoss contributions independently. While the skeptic's concern about whether the semantic content of binary codes matters versus extra learnable tokens is a valid correctness risk, it is not a circularity issue—the paper does not claim a prediction that reduces to its inputs by construction.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new physical entities, particles, or forces. The Semantic Relation Encoding (SRE) and Semantic Condensation loss (ScLoss) are algorithmic constructs built from standard mathematical operations (cosine similarity, random projection, SVD) applied to CLIP's existing embedding space. The free parameters (λ, τ, d_free, M, σ) are standard hyperparameters for prompt tuning methods, with λ explicitly inherited from KgCoOp. The core axioms are domain assumptions about the semantic content of CLIP embeddings and the noise structure of singular values, both of which are plausible but not independently verified.

free parameters (5)
  • λ (loss weight) = 8.0
    Controls ScLoss contribution; set to 8.0 matching KgCoOp, tested over {1,2,4,8,16} with <1% variation
  • τ (SVD threshold) = 0.3
    Threshold for singular value filtering in ScLoss; fixed across all datasets, tested from 0.1-0.5 with <0.3% variation
  • d_free = 3
    Additional free space in projection dimension d=ceil(log2(N)+d_free); tested 0-8, best at 3 with 0.36% range
  • M (context length) = 4
    Number of learnable context vectors; tested {1,2,4,8,16}, 4 used in main experiments
  • σ (Gaussian init std) = 0.02
    Standard deviation for context vector initialization
assumptions (3)
  • domain assumption CLIP's text embeddings of class names encode meaningful taxonomic relationships that can be preserved through signed random projection
    Invoked in §3.2.1 to justify SRE; the paper states 'CLIP's text embeddings inherently encode taxonomic relationships' without empirical verification of this claim for fine-grained categories
  • domain assumption Small singular values of the handcrafted prompt embedding matrix correspond to noise rather than fine-grained semantic signal
    Invoked in §3.2.2-3.2.3 to justify SVD truncation; motivated by Marchenko-Pastur theorem for pure noise matrices, but the actual embedding matrix is not pure noise
  • ad hoc to paper Prepending binary semantic relation codes to learnable prompt tokens allows the text encoder to utilize inter-class structure
    Invoked in §3.2.1 Eq.(7); the mechanism by which the text encoder processes binary codes as word embeddings is not empirically validated beyond end-to-end accuracy gains

how reviews work

0 comments
Cite this review

Pith. "Pith review of Structured-Condensed Prompt Tuning in Vision-Language Models for Fine-grained Image Recognition." pith.science (2026). https://pith.science/paper/XKQDBCID

@misc{pith2026260706185,
  author       = {Pith},
  title        = {Pith review of: Structured-Condensed Prompt Tuning in Vision-Language Models for Fine-grained Image Recognition},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XKQDBCID}},
  note         = {Machine review of arXiv:2607.06185}
}
read the original abstract

Fine-grained image recognition poses a significant challenge due to the substantial expertise and effort required for manual annotation. Vision-language models (VLMs) like CLIP provide a compelling zero-shot alternative, reducing reliance on extensive labeled data. However, their ability to capture subtle distinctions remains limited, leading to subpar recognition performance. While prompt tuning has proven effective for adapting VLMs, most existing methods treat class labels as isolated, discrete entities, overlooking the rich semantic relationships between them. This oversimplified assumption limits the model's ability to capture hierarchical dependencies and inter-class correlations -- both critical for distinguishing visually similar categories. The problem is especially acute in fine-grained classification, where accurate recognition depends on understanding complex label semantics. To address this, we propose Structured-Condensed Prompt Tuning (SCPT), which enhances semantic structure modeling in prompt learning. Specifically, we introduce Semantic Relation Encoding (SRE) to explicitly model inter-class semantic topology and encode structured label relationships. In parallel, we design a Semantic Condensation loss (ScLoss) to suppress redundant supervision and extract discriminative components from the global semantic space. Together, these components significantly improve semantic alignment and fine-grained discrimination. Extensive experiments on 14 fine-grained benchmarks show that SCPT effectively mitigates semantic ambiguity and achieves state-of-the-art performance in both few-shot and base-to-novel generalization settings.

Figures

Figures reproduced from arXiv: 2607.06185 by the authors.

Figure 1
Figure 1. From static to semantically structured prompts: a paradigm shift in prompting mechanisms. classification without task-specific training. While VLMs demonstrate strong generalization in broad categories, this advantage diminishes notably in fine-grained recognition contexts, predominantly stemming from the model’s constrained capacity to discern nuanced inter-class variations. Prompt tuning adapts task-specific promp… view at source ↗
Figure 2
Figure 2. Overview of the proposed Structured-Condensed Prompt Tuning (SCPT) framework. The proposed method is grounded in the observation that employing isolated, discrete category 4 [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Comparative Analysis of SVD Dropout Efficacy: Accuracy Dynamics and Singular Value Distribution. (a)Accuracy variation across three datasets under varying principal component retention levels K. Dashed red lines denote baseline performance without SVD Dropout. (b)Post-decomposition singular value magnitude distribution of semantic embeddings. To explore this question, we visualize the distribution of nonzero singula… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Impact of additional free space df ree. Effect of dfree in SRE. This study examines the impact of the additional free space dfree on encoding length and recognition accuracy in a 16-shot setting across 14 datasets, as shown in [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]
Figure 5
Figure 5. Figure 5: Effect of learnable prompt Length M. 16, finding that SRE-encoded prompts are largely insensitive to learnable parameter variations. While a prompt length of 2 yields the highest accuracy, the difference from the lowest result is just 0.16%. Shorter prompt lengths gene…
Figure 6
Figure 6. Figure 6: Effect of loss weight λ. Effect of Threshold τ . The threshold τ controls the strength of feature noise filtering. We evaluate the sensitivity to τ by varying its value from 0.1 to 0.5 under the same experimental setting, using the ViT-B/32 backbone on Fruit92, FGVC-Ai…
Figure 7
Figure 7. Figure 7: Effect of threshold τ . signed random projection strategy is insensitive to initialization noise, validating its reliability for consistent deployment [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 8
Figure 8. Figure 8: Comparison of visual attention heatmaps generated by different methods: (a) Ground-truth images; (b) Heatmaps from CoOp similarity scores; (c) Heatmaps from KgCoOp similarity scores; (d) Heatmaps from SCPT similarity scores. Textual embeddings act as explicit superviso…
Figure 9
Figure 9. Figure 9: t-SNE visualizations of visual embeddings on three datasets. The Silhouette Coefficient (S) is reported to quantify cluster compactness and separation. gap between SCPT and CoOp becomes more pronounced as the number of categories increases, indicating that SCPT is effe…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

52 extracted references · 52 canonical work pages

  1. [1]

    Zero-shot sketch-based image retrieval via adaptive relation-aware metric learning.Pattern Recognition, 152:110452, 2024

    Yang Liu, Yuhao Dang, Xinbo Gao, Jungong Han, and Ling Shao. Zero-shot sketch-based image retrieval via adaptive relation-aware metric learning.Pattern Recognition, 152:110452, 2024. ISSN 0031-3203

    work page 2024

  2. [2]

    Complementary two-branch transformer for multi-label image retrieval.Pattern Recognition, 168:111806, 2025

    Ying Li, Shuaiyu Deng, Chunming Guan, and Jiaquan Gao. Complementary two-branch transformer for multi-label image retrieval.Pattern Recognition, 168:111806, 2025. ISSN 0031-3203

    work page 2025

  3. [3]

    Changes to captions: An attentive network for remote sensing change captioning.IEEE Transactions on Image Processing, 32:6047–6060, 2023

    Shizhen Chang and Pedram Ghamisi. Changes to captions: An attentive network for remote sensing change captioning.IEEE Transactions on Image Processing, 32:6047–6060, 2023

    work page 2023

  4. [4]

    Reparameterizing and dynamically quantizing image features for image generation.Pattern Recognition, 146:109962, 2024

    Mingzhen Sun, Weining Wang, Xinxin Zhu, and Jing Liu. Reparameterizing and dynamically quantizing image features for image generation.Pattern Recognition, 146:109962, 2024. ISSN 0031-3203

    work page 2024

  5. [5]

    Txt2img-mhn: Remote sensing image generation from text using modern hopfield networks.IEEE Transactions on Image Processing, 32:5737–5750, 2023

    Yonghao Xu, Weikang Yu, Pedram Ghamisi, Michael Kopp, and Sepp Hochreiter. Txt2img-mhn: Remote sensing image generation from text using modern hopfield networks.IEEE Transactions on Image Processing, 32:5737–5750, 2023

    work page 2023

  6. [6]

    Fine grained food image recognition based on swin transformer.Journal of Food Engineering, 380:112134, 2024

    Zhiyong Xiao, Guang Diao, and Zhaohong Deng. Fine grained food image recognition based on swin transformer.Journal of Food Engineering, 380:112134, 2024. ISSN 0260-8774

    work page 2024

  7. [7]

    Convolution-enhanced bi-branch adaptive transformer with cross-task interaction for food category and ingredient recognition

    Yuxin Liu, Weiqing Min, Shuqiang Jiang, and Yong Rui. Convolution-enhanced bi-branch adaptive transformer with cross-task interaction for food category and ingredient recognition. IEEE Transactions on Image Processing, 33:2572–2586, 2024

    work page 2024

  8. [8]

    Synthesizing knowledge-enhanced features for real-world zero-shot food detection.IEEE Transactions on Image Processing, 33:1285–1298, 2024

    Pengfei Zhou, Weiqing Min, Jiajun Song, Yang Zhang, and Shuqiang Jiang. Synthesizing knowledge-enhanced features for real-world zero-shot food detection.IEEE Transactions on Image Processing, 33:1285–1298, 2024

    work page 2024

Show all 52 references
  1. [9]

    Food recommendation towards personalized wellbeing.Trends in Food Science & Technology, 156:104877, 2025

    Guanhua Qiao, Dachuan Zhang, Nana Zhang, Xiaotao Shen, Xidong Jiao, Wenwei Lu, Daming Fan, Jianxin Zhao, Hao Zhang, Wei Chen, and Jinlin Zhu. Food recommendation towards personalized wellbeing.Trends in Food Science & Technology, 156:104877, 2025. ISSN 0924-2244

    2025

  2. [10]

    Large scale visual food recognition.IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(8):9932–9949, 2023

    Weiqing Min, Zhiling Wang, Yuxin Liu, Mengjiang Luo, Liping Kang, Xiaoming Wei, Xiaolin Wei, and Shuqiang Jiang. Large scale visual food recognition.IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(8):9932–9949, 2023

    2023

  3. [11]

    Instance-level few-shot learning with class hierarchy mining.IEEE Transactions on Image Processing, 32:2374–2385, 2023

    Anh-Khoa Nguyen Vu, Thanh-Toan Do, Nhat-Duy Nguyen, Vinh-Tiep Nguyen, Thanh Duc Ngo, and Tam V Nguyen. Instance-level few-shot learning with class hierarchy mining.IEEE Transactions on Image Processing, 32:2374–2385, 2023

    2023

  4. [12]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. InInternational conference on machine learning, pag...

    2021

  5. [13]

    Learning to prompt for vision-language models.International Journal of Computer Vision, 130(9):2337–2348, 2022

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to prompt for vision-language models.International Journal of Computer Vision, 130(9):2337–2348, 2022. 21

    2022

  6. [14]

    Conditional prompt learning for vision-language models

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Conditional prompt learning for vision-language models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16816–16825, 2022

    2022

  7. [15]

    Scaling up visual and vision-language representation learning with noisy text supervision

    Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representation learning with noisy text supervision. InInternational conference on machine learning, pages 4904–4916. ...

    2021

  8. [16]

    Blip: Bootstrapping language-image pre- training for unified vision-language understanding and generation

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image pre- training for unified vision-language understanding and generation. InInternational conference on machine learning, pages 12888–12900. PMLR, 2022

    2022

  9. [17]

    Effective conditioned and composed image retrieval combining clip-based features

    Alberto Baldrati, Marco Bertini, Tiberio Uricchio, and Alberto Del Bimbo. Effective conditioned and composed image retrieval combining clip-based features. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21466–21474, 2022

    2022

  10. [18]

    Clip for all things zero-shot sketch-based image retrieval, fine-grained or not

    Aneeshan Sain, Ayan Kumar Bhunia, Pinaki Nath Chowdhury, Subhadeep Koley, Tao Xiang, and Yi-Zhe Song. Clip for all things zero-shot sketch-based image retrieval, fine-grained or not. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 276...

    2023

  11. [19]

    Cris: Clip-driven referring image segmentation

    Zhaoqing Wang, Yu Lu, Qiang Li, Xunqiang Tao, Yandong Guo, Mingming Gong, and Tongliang Liu. Cris: Clip-driven referring image segmentation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11686–11695, 2022

    2022

  12. [20]

    Image segmentation using text and image prompts

    Timo Lüddecke and Alexander Ecker. Image segmentation using text and image prompts. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7086–7096, 2022

    2022

  13. [21]

    Sedigheh Eslami, Christoph Meinel, and Gerard De Melo. Pubmedclip: How much does clip benefit visual question answering in the medical domain? InFindings of the Association for Computational Linguistics: EACL 2023, pages 1181–1193, 2023

    2023

  14. [22]

    Maple: Multi-modal prompt learning

    Muhammad Uzair Khattak, Hanoona Rasheed, Muhammad Maaz, Salman Khan, and Fa- had Shahbaz Khan. Maple: Multi-modal prompt learning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19113–19122, 2023

    2023

  15. [23]

    Self-regulating prompts: Foundational model adaptation without forgetting

    Muhammad Uzair Khattak, Syed Talal Wasim, Muzammal Naseer, Salman Khan, Ming-Hsuan Yang, and Fahad Shahbaz Khan. Self-regulating prompts: Foundational model adaptation without forgetting. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 15190–15...

    2023

  16. [24]

    Dgprompt: Dual-guidance prompts generation for vision-language models.Neural Networks, page 107472, 2025

    Tai Zheng, Zhen-Duo Chen, Zi-Chao Zhang, Zhen-Xiang Ma, Li-Jun Zhao, Chong-Yu Zhang, Xin Luo, and Xin-Shun Xu. Dgprompt: Dual-guidance prompts generation for vision-language models.Neural Networks, page 107472, 2025

    2025

  17. [25]

    Prompt distribution learning

    Yuning Lu, Jianzhuang Liu, Yonggang Zhang, Yajing Liu, and Xinmei Tian. Prompt distribution learning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5206–5215, 2022. 22

    2022

  18. [26]

    Visual-language prompt tuning with knowledge- guided context optimization

    Hantao Yao, Rui Zhang, and Changsheng Xu. Visual-language prompt tuning with knowledge- guided context optimization. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6757–6767, 2023

    2023

  19. [27]

    Tcp: Textual-based class-aware prompt tuning for visual-language model

    Hantao Yao, Rui Zhang, and Changsheng Xu. Tcp: Textual-based class-aware prompt tuning for visual-language model. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 23438–23448, 2024

    2024

  20. [28]

    Bayesian test-time adaptation for vision-language models

    Lihua Zhou, Mao Ye, Shuaifeng Li, Nianxin Li, Xiatian Zhu, Lei Deng, Hongbin Liu, and Zhen Lei. Bayesian test-time adaptation for vision-language models. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 29999–30009, 2025

    2025

  21. [29]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at...

    2021

  22. [30]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016

    2016

  23. [31]

    Long-clip: Unlocking the long-text capability of clip

    Beichen Zhang, Pan Zhang, Xiaoyi Dong, Yuhang Zang, and Jiaqi Wang. Long-clip: Unlocking the long-text capability of clip. InEuropean Conference on Computer Vision, pages 310–325. Springer, 2024

    2024

  24. [32]

    Extensions of lipschitz mappings into a hilbert space.Contemporary mathematics, 26(189-206):1, 1984

    William B Johnson, Joram Lindenstrauss, et al. Extensions of lipschitz mappings into a hilbert space.Contemporary mathematics, 26(189-206):1, 1984

    1984

  25. [33]

    A short proof of the marchenko–pastur theorem.Comptes Rendus Mathematique, 354(3):319–322, 2016

    Pavel Yaskov. A short proof of the marchenko–pastur theorem.Comptes Rendus Mathematique, 354(3):319–322, 2016

    2016

  26. [34]

    A new dataset of dog breed images and a benchmark for finegrained classification.Computational Visual Media, 6:477–487, 2020

    Ding-Nan Zou, Song-Hai Zhang, Tai-Jiang Mu, and Min Zhang. A new dataset of dog breed images and a benchmark for finegrained classification.Computational Visual Media, 6:477–487, 2020

    2020

  27. [35]

    Cats and dogs

    Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and CV Jawahar. Cats and dogs. In 2012 IEEE conference on computer vision and pattern recognition, pages 3498–3505. IEEE, 2012

    2012

  28. [36]

    Automated flower classification over a large number of classes

    Maria-Elena Nilsback and Andrew Zisserman. Automated flower classification over a large number of classes. In2008 Sixth Indian conference on computer vision, graphics & image processing, pages 722–729. IEEE, 2008

    2008

  29. [37]

    3d object representations for fine-grained categorization

    Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3d object representations for fine-grained categorization. InProceedings of the IEEE international conference on computer vision workshops, pages 554–561, 2013

    2013

  30. [38]

    Webly supervised fine-grained recognition: Benchmark datasets and an approach

    Zeren Sun, Yazhou Yao, Xiu-Shen Wei, Yongshun Zhang, Fumin Shen, Jianxin Wu, Jian Zhang, and Heng Tao Shen. Webly supervised fine-grained recognition: Benchmark datasets and an approach. InProceedings of the IEEE/CVF international conference on computer vision, pages 10602–106...

    2021

  31. [39]

    Novel dataset for fine-grained image categorization

    Aditya Khosla, Nityananda Jayadevaprakash, Bangpeng Yao, and Li Fei-Fei. Novel dataset for fine-grained image categorization. InFirst Workshop on Fine-Grained Visual Categorization, IEEE Conference on Computer Vision and Pattern Recognition, Colorado Springs, CO, June 2011

    2011

  32. [40]

    J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. ImageNet: A Large-Scale Hierarchical Image Database. InCVPR09, 2009

    2009

  33. [41]

    Vegfru: A domain-specific dataset for fine-grained visual categorization

    Saihui Hou, Yushan Feng, and Zilei Wang. Vegfru: A domain-specific dataset for fine-grained visual categorization. InProceedings of the IEEE international conference on computer vision, pages 541–549, 2017

    2017

  34. [42]

    The caltech-ucsd birds-200-2011 dataset

    Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech-ucsd birds-200-2011 dataset. California Institute of Technology, 2011

    2011

  35. [43]

    Food-101–mining discriminative components with random forests

    Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. Food-101–mining discriminative components with random forests. InComputer vision–ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceedings, part VI 13, pages 446–461. Springer, 2014

    2014

  36. [44]

    Deep-based ingredient recognition for cooking recipe retrieval

    Jingjing Chen and Chong-Wah Ngo. Deep-based ingredient recognition for cooking recipe retrieval. InProceedings of the 24th ACM international conference on Multimedia, pages 32–41, 2016

    2016

  37. [45]

    Ingredient-guided cascaded multi-attention network for food recognition

    Weiqing Min, Linhu Liu, Zhengdong Luo, and Shuqiang Jiang. Ingredient-guided cascaded multi-attention network for food recognition. InProceedings of the 27th ACM International Conference on Multimedia, pages 1331–1339, 2019

    2019

  38. [46]

    Isia food-500: A dataset for large-scale food recognition via stacked global-local attention network

    Weiqing Min, Linhu Liu, Zhiling Wang, Zhengdong Luo, Xiaoming Wei, Xiaolin Wei, and Shuqiang Jiang. Isia food-500: A dataset for large-scale food recognition via stacked global-local attention network. InProceedings of the 28th ACM International Conference on Multimedia, pages...

    2020

  39. [47]

    Fine- grained visual classification of aircraft, 2013

    Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew Blaschko, and Andrea Vedaldi. Fine- grained visual classification of aircraft, 2013. arXiv:1306.5151

    2013 arXiv

  40. [48]

    Learning to prompt with text only supervision for vision-language models

    Muhammad Uzair Khattak, Muhammad Ferjad Naeem, Muzammal Naseer, Luc Van Gool, and Federico Tombari. Learning to prompt with text only supervision for vision-language models. InProceedings of the AAAI Conference on Artificial Intelligence, pages 4230–4238, 2025

    2025

  41. [49]

    Advancing textual prompt learning with anchored attributes

    Zheng Li, Yibing Song, Ming-Ming Cheng, Xiang Li, and Jian Yang. Advancing textual prompt learning with anchored attributes. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 3618–3627, 2025

    2025

  42. [50]

    Grad-cam: Visual explanations from deep networks via gradient-based localization

    Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. InProceedings of the IEEE international conference on computer vision, pages 618–626, 2017

    2017

  43. [51]

    Visual prompt tuning

    Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, and Ser-Nam Lim. Visual prompt tuning. InEuropean conference on computer vision, pages 709–727. Springer, 2022. 24

    2022

  44. [52]

    Dualprompt: Complementary prompting for rehearsal-free continual learning

    Zifeng Wang, Zizhao Zhang, Sayna Ebrahimi, Ruoxi Sun, Han Zhang, Chen-Yu Lee, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, et al. Dualprompt: Complementary prompting for rehearsal-free continual learning. InEuropean conference on computer vision, pages 631–648. Springer...

    2022

Pith tools

Reviewed July 8, 2026 · model on record in the stance chip above.