{"id":"71cae1c2-42ee-42d2-8d8b-f3a067a22348","arxiv_id":"2508.10645","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"SemPT, a semantic prompt tuning method, uses shared attribute-level descriptions and adaptive embedding selection to slightly improve base-to-novel, cross-dataset, cross-domain, and few-shot generalization of CLIP prompt tuning.","lead":"SemPT improves vision-language model transfer by generating shared attribute descriptions with LLMs, weighting them by visual relevance, and using different text embeddings for seen versus unseen classes. The method is tested on 15 benchmarks and shows small but consistent gains over strong prompt-tuning baselines like PromptKD.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Novel-class names leak into training via LLM prompting and loss denominators, so 'unseen' generalization gains may be transductive.","rationale":"The reader's weakest assumption focuses on the lack of verification of the shared-attribute vocabulary. While that is a valid reproducibility concern, the most load-bearing issue is more specific: the method's knowledge-construction step and training objectives explicitly use novel-class names and their LLM-generated descriptions, so the evaluation may violate the intended 'unseen' protocol. This transductive leakage is not independently controlled in any ablation; the 'category names only' baseline also uses all N class names. I partially agree with the reader because they noted the use of category names from both seen and unseen classes but did not draw out the implication that this leaks test-class identity into training. The beta inconsistency (Section 4.3 says β=0.4; Table 4(d) says optimal β=0.6) is a further concrete reproducibility issue, but it is secondary to the validity threat. The verdict remains CONDITIONAL: the paper could still be acceptable if the inductive re-run preserves the gains, but as written the SOTA claim is not yet established. Hence UNCHANGED relative to the reader's conditional verdict.","tokens_in":19098,"tokens_out":8734,"duration_ms":104087,"concrete_test":"Re-run the base-to-novel experiment in a strictly inductive setting: prompt the LLM using only the base/seen class names (for both attribute discovery and description generation), restrict the training-loss denominators in Eqs. 12–14 to seen classes only, and at inference construct Tenh for novel classes using descriptions generated from the seen-only attribute vocabulary. Compare novel-class accuracy/HM against the reported 81.21 novel and 84.04 HM. If HM drops by more than ~1 point or falls at/below PromptKD's 83.73, the claimed improvement is substantially attributable to transductive use of novel class names.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim of generalizing to unseen categories is threatened by transductive leakage of novel/unseen class names during training. Section 3.1 defines N = N_s + N_u and says labels are drawn from seen or unseen categories; Section 3.2 (Steps 1 and 2) explicitly prompts the LLM with category examples from both Cs and Cu and generates attribute-level descriptions for all ci ∈ Cs ∪ Cu. These unseen-class text embeddings are then used in the training losses (Eqs. 12–14): although training images have base-class labels, the softmax denominators sum over all N classes, so the model is trained to push image embeddings away from unseen-class label and attribute-enhanced embeddings. The shared attribute vocabulary A is also extracted using both seen and unseen names. Thus the 'novel' classes are not truly unseen during training—their names and LLM-generated descriptions shape the embedding space. The reported HM gains (e.g., PromptKD 83.73 → SemPT 84.04) could stem from this transductive exposure rather than from the proposed shared-attribute transfer mechanism. The ablation 'category names only' (83.83 HM) also uses all N names, so it does not control for this. Without an experiment that removes unseen names from knowledge construction and training objectives, the SOTA claim is not established.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"SemPT is a prompt-tuning framework for CLIP-based vision-language models. The paper proposes to build a shared attribute vocabulary by prompting an LLM in two steps: first extract cross-category visual attributes from both seen and unseen class names, then generate attribute-level descriptions for every class. These descriptions are encoded, visually weighted by top-K selection with softmax attention, fused with label text embeddings via an MLP and residual connection, and trained with two softmax losses. At inference, label embeddings are used for seen classes and attribute-enhanced embeddings for unseen classes. The method is evaluated on 15 datasets under base-to-novel, cross-dataset, cross-domain, and few-shot settings, integrated with PromptKD and MMRL baselines. The authors report consistent but small improvements over these baselines, e.g., PromptKD w/SemPT reaches 84.04% harmonic mean in base-to-novel, 72.11% in cross-dataset, and 72.05% in cross-domain.","tokens_in":19455,"tokens_out":5213,"duration_ms":60666,"significance":"If validated, SemPT would offer a simple plug-in enhancement for existing prompt-tuning methods by explicitly modeling shared attribute-level semantics, and the adaptive seen/unseen inference rule is a sensible design. The paper is reasonably positioned relative to LLM-augmented prompt tuning (HPT, ArGue, CoCoLe, ProText) and tests a broad set of benchmarks with two strong baselines. However, the reported gains are modest (0.3–0.8 points), and the evidence is weakened by the fact that unseen class names and descriptions are used during training, by a concrete hyperparameter inconsistency, by the absence of error bars and significance tests, and by the lack of prompt templates or LLM outputs needed for reproducibility. The central claim of transfer via shared attributes is therefore not yet established.","major_comments":[{"comment":"The framework uses unseen-class names and attribute descriptions during training. Sec. 3.1 defines N = N_s + N_u, and Sec. 3.2 prompts the LLM with category examples from both C_s and C_u, producing attribute vocabulary A and descriptions T for all categories. The training losses in Eqs. (12)–(14) normalize over all N categories, so image embeddings are pushed away from unseen-class label and attribute-enhanced text embeddings. Thus the 'unseen' categories are not truly unseen at training time. The 'category names only' ablation in Table 4(b) also uses all N names, so it does not control for this. The reported gains in Table 1 could stem from transductive exposure to unseen class names rather than from the proposed shared-attribute transfer. Please add a control experiment that removes C_u from the LLM knowledge construction and from the loss denominators, and report base/novel HM. If th","section":"§3.1–3.2 and Eqs. (12)–(14)"},{"comment":"There is a direct contradiction in the loss-balancing hyperparameter. Implementation details (Sec. 4.3) state that β in Eq. (14) is set to 0.4. The ablation in Table 4(d) reports that the optimal β is 0.6, achieving 84.04% HM, which equals the main result reported in Table 1. It is therefore unclear which β was used for the final results. If β=0.6 was used, the implementation statement is wrong; if β=0.4 was used, the ablation's conclusion is unsupported. This must be corrected and the value of β for each experimental setting must be specified explicitly.","section":"§4.3 vs Table 4(d)"},{"comment":"All results are reported as single numbers without variance, multiple seeds, or significance tests. The headline improvements are small (0.31–0.78 percentage points) and some per-dataset values decrease (e.g., ImageNet base drops from 80.83 to 80.71 in Table 1). Without error bars, it is not possible to tell whether the average gains are meaningful. Please report mean±std over at least three seeds and, where feasible, paired significance tests across the datasets.","section":"Tables 1–3 and Fig. 4"},{"comment":"The two-step prompting strategy is the core of the method, but the exact prompt templates and LLM outputs are not provided. No examples of the shared attribute vocabulary A or of the generated descriptions T are shown, and there is no manual or automatic verification that the attributes are shared or visually meaningful. This makes the method non-reproducible and leaves the central assumption of transferable semantic bridges unverified. Please include the full prompts in an appendix, provide sample outputs per dataset, and ideally add a human or automated consistency check.","section":"§3.2 Knowledge Construction"}],"minor_comments":[{"comment":"In the FGVCAircraft column, KIM reports Base=42.80, Novel=37.70, but HM=30.10. The harmonic mean of those two numbers is about 40.1, suggesting a typo. Please check.","section":"Table 1, KIM row"},{"comment":"There is a typo in 'Tlbl∈ R N N×d'; it should be 'T_lbl ∈ R^{N×d}'.","section":"§3.1"},{"comment":"The 'one-step prompting strategy' is not defined. Please specify how it differs from the two-step strategy (e.g., no explicit attribute vocabulary) so the ablation is interpretable.","section":"Table 4(b)"},{"comment":"The statement that all baselines were 'faithfully reproduced' would be easier to verify if code/checkpoints or a detailed reproduction protocol were provided. Consider adding a footnote with repository links or a reproducibility statement.","section":"§4.3"}],"recommendation":"major_revision","confidential_remarks":"The paper is a modest empirical contribution whose main claim depends on excluding transductive leakage and on reporting statistics. If the authors can supply the control experiment, correct the β inconsistency, and add error bars, the work may become acceptable for a journal. The lack of LLM prompts and outputs is a further barrier to reproducibility and should be treated as a required revision, not an optional extra."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea here is sensible: instead of relying on sparse labels or disparate LLM descriptions, extract a shared attribute vocabulary and generate attribute-level descriptions, then weight them by visual relevance and fuse with label embeddings. The plug-in compatibility with PromptKD and MMRL is a genuinely nice property, and the paper shows consistent small gains across many datasets and settings. The two-step prompting plus visual weighting plus dual-embedding inference is a real combination that I don't think appears in the cited prior work. Credit where due: the experiments are extensive, the ablations cover the main components, and the writing is clear.\n\nThe soft spots are also clear. Most importantly, the stress-test concern lands. Section 3.2 explicitly prompts the LLM with category examples from both Cs and Cu, and the training losses in Eqs. 12–14 sum over all N categories, including unseen ones. So the model is trained to push image embeddings away from unseen-class label and attribute-enhanced embeddings. That means the 'novel' classes are not truly unseen during training. The ablation 'category names only' also uses all N names, so it does not control for this. Without an experiment that removes unseen names from knowledge construction and training objectives, the SOTA claim is not established as a genuine generalization result. The gains could largely stem from transductive exposure to novel class names. This is a load-bearing flaw for the paper's central claim.\n\nThere's also a concrete reporting error: beta is stated as 0.4 in the Implementation Details but the ablation shows the optimal beta is 0.6, and the final results presumably use 0.6. That inconsistency needs fixing. More generally, there are no error bars, no multiple seeds, and no significance tests, which matters because the gains are small (0.3–0.8 points). And there is no code or data, nor any example LLM outputs or manual inspection of the extracted attributes, so the qualitative claim that shared attributes form useful semantic bridges is unverified.\n\nWho is this for? Researchers working on prompt tuning for VLMs, particularly those exploring LLM-generated descriptions. It's a useful engineering contribution that could become a solid paper after the transductive leakage is addressed and the reporting is fixed. The paper deserves a serious referee rather than a desk reject, but the referee should demand the controlled experiment that isolates the attribute-sharing mechanism from the mere presence of unseen class names, plus variance estimates and code release.\n\nMy recommendation: send it to peer review, but expect substantial revision.","headline":"Useful plug-in method with consistent small gains, but the 'unseen' generalization claim is compromised by transductive leakage of novel class names into the LLM prompting and training losses.","tokens_in":19849,"tokens_out":2545,"would_cite":false,"duration_ms":28644,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SemPT claims that shared attribute-level descriptions generated by a two-step LLM prompt build transferable semantic bridges that let vision-language prompt tuning generalize to unseen categories, and reports state-of-the-art results across","keywords":["prompt tuning","vision-language models","transfer learning","attribute-level descriptions","large language models","zero-shot generalization","base-to-novel generalization","CLIP"],"falsifier":"Run the exact SemPT setup on one of the 11 standard benchmarks but replace the shared attribute vocabulary with a matched set of per-category attributes generated independently (no shared vocabulary), keeping description count, fusion, and training identical; if base-to-novel harmonic mean does not drop meaningfully, the shared-attribute claim is unsupported. A second check is to read the two-step LLM outputs: if unseen categories share few or no attributes with seen categories, the semantic-bridge mechanism cannot be carrying the reported gains.","tokens_in":19064,"feed_emoji":"🧩","tokens_out":9997,"duration_ms":93906,"temperature":0.7,"pith_summary":"SemPT claims that prompt tuning for vision-language models fails to generalize to unseen classes because text embeddings are fragmented: category labels are too sparse and LLM-generated descriptions are too disparate to form connections across categories. The paper's remedy is to build a shared vocabulary of visual attributes with a two-step LLM prompt and to express every category, seen and unseen, as combinations of those attributes, creating explicit semantic bridges from the known to the unknown. It then weights attribute descriptions by visual relevance, fuses them into label embeddings through a residual projection, and trains label and attribute-enhanced objectives jointly. At inference, seen classes use label embeddings while unseen classes use attribute-enhanced embeddings. Across 15 benchmarks, the approach reports the best harmonic-mean results in base-to-novel, cross-dataset, and cross-domain settings when plugged into two existing prompt-tuning methods.","feed_headline":"Shared attribute vocabulary beats prior CLIP prompt tuning","feed_subtitle":"Two-step LLM prompts mine shared attributes, giving gains on 15 benchmarks.","key_machinery":"The central object is the shared attribute vocabulary $\\mathcal{A} = \\{a_m\\}_{m=1}^{M_a}$, produced by the first LLM prompt over both seen and unseen category names, together with the attribute-level description set $\\mathcal{T} = \\{t_i^j\\}$ generated in a second prompt conditioned on $\\mathcal{A}$. The shared vocabulary is what makes the space connected: unseen categories can be decomposed into attribute combinations already present in seen categories. The mechanism that makes the vocabulary useful is the visually guided weighting (top-K selection with temperature-scaled softmax over image–description cosine similarities) and the residual fusion $T_{\\text{enh}}^{(i)} = (1-\\$\\alpha$) T_{\\text{l","core_discovery":"The central claim is that shared attribute-level knowledge, rather than category names or free-form LLM descriptions, is the right carrier of transferable semantics for vision-language prompt tuning. The paper argues that embeddings built from labels alone or from unrelated descriptions form isolated clusters with no semantic path to unseen categories. SemPT's two-step prompting first asks an LLM for a compact vocabulary of visual attributes that applies across all categories, then asks it to express each category as short descriptions grounded in that vocabulary. The resulting attribute-aligned text embeddings are selected per image by top-K cosine similarity with temperature-scaled softmax","pith_inferences":["A direct consequence the paper leaves implicit: if shared attributes drive transfer, then unseen categories expressible as combinations of known attributes should be recognizable with zero labeled images, which can be tested by probing with synthetic attribute text.","The dependence on the specific LLM and prompt wording is unexplored; a natural extension is to vary the LLM and measure whether vocabulary agreement across LLMs predicts transfer performance.","The binary seen/unseen inference switch could be softened into a confidence-based blend of label and attribute embeddings, a variation the paper does not test but its dual-embedding view suggests.","The dual-embedding design could be adapted to class-incremental learning, where the seen set grows over time and attribute-enhanced embeddings could preserve old-class discrimination while extending to new classes."],"forward_implications":["SemPT can be layered onto existing prompt-tuning methods (PromptKD, MMRL) and reports consistent gains over those baselines across all four evaluation settings.","Improvements are strongest on fine-grained datasets like StanfordCars and OxfordPets, where attribute-level descriptions help separate visually similar classes.","The ablation shows the mixed inference rule (labels for seen, attribute-enhanced for unseen) beats using either embedding type for all categories, so category exposure should guide text representation choice.","Two-step prompting with a shared vocabulary outperforms both one-step prompting and category names alone, indicating the structure of LLM knowledge construction matters beyond description content.","Because only prompts and a small MLP are trained, SemPT retains few-shot compatibility, where it reports the best 16-shot average among compared methods."],"supporting_citations":[{"why":"Supplies the frozen CLIP dual-encoder space in which all text and image embeddings are computed and compared.","marker":"[1]"},{"why":"CoOp is the label-based prompt-tuning method whose sparse-label limitation SemPT targets.","marker":"[5]"},{"why":"CoCoOp is the dynamic-prompt baseline that SemPT compares against and improves on.","marker":"[6]"},{"why":"PromptKD is the main baseline SemPT is plugged into; the headline results are PromptKD w/SemPT versus PromptKD.","marker":"[12]"},{"why":"HPT generates hierarchical LLM descriptions, representing the disparate-description approach SemPT argues is fragmented.","marker":"[13]"},{"why":"CoPrompt enforces consistency with LLM features and is another LLM-augmented baseline in the comparisons.","marker":"[14]"},{"why":"ArGue is the attribute-guided prompt-tuning method closest to SemPT; the paper contrasts its own shared vocabulary with ArGue's attribute injection.","marker":"[15]"},{"why":"CoCoLe distills conceptual knowledge into a codebook, a competing LLM-knowledge representation SemPT compares against.","marker":"[16]"},{"why":"MMRL is the second baseline SemPT is integrated with, used to show broad compatibility and to report few-shot results.","marker":"[24]"}],"fun_headline_variants":["Two-step LLM prompting extracts shared visual attributes","Attribute-aware prompts let SemPT top 15 benchmarks","Shared attribute embeddings bridge seen to unseen categories","SemPT: attribute-level tuning beats label-only for CLIP","LLM-mined shared attributes enhance vision-language transfer"],"cache_read_input_tokens":2816,"weakest_assumption_plain":"The load-bearing premise is that an LLM, given the names of both seen and unseen categories and told to identify shared visual attributes, will produce a vocabulary that genuinely transfers across the seen–unseen boundary; the paper provides no LLM outputs, prompts, or manual inspection to verify that the shared attributes are real, distinct, and useful.","fun_headline_variants_meta":{"raw":{"variants":["Two-step LLM prompting extracts shared visual attributes","Attribute-aware prompts let SemPT top 15 benchmarks","Shared attribute embeddings bridge seen to unseen categories","SemPT: attribute-level tuning beats label-only for CLIP","LLM-mined shared attributes enhance vision-language transfer"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000332,"raw_usage":{"total_tokens":1684,"prompt_tokens":745,"completion_tokens":939,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":489,"completion_tokens_details":{"reasoning_tokens":874}},"tokens_in":489,"tokens_out":939,"duration_ms":10373,"temperature":1.0,"reasoning_tokens":874,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T20:17:49.048635+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the exact SemPT setup on one of the 11 standard benchmarks but replace the shared attribute vocabulary with a matched set of per-category attributes generated independently (no shared vocabulary), keeping description count, fusion, and training identical; if base-to-novel harmonic mean does not drop meaningfully, the shared-attribute claim is unsupported. A second check is to read the two-step LLM outputs: if unseen categories share few or no attributes with seen categories, the semantic-bridge mechanism cannot be carrying the reported gains.","supporting_citations":[{"cited_title":"Radford, J","cited_arxiv_id":null,"evidence_quote":"Supplies the frozen CLIP dual-encoder space in which all text and image embeddings are computed and compared."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"CoOp is the label-based prompt-tuning method whose sparse-label limitation SemPT targets."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"CoCoOp is the dynamic-prompt baseline that SemPT compares against and improves on."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"PromptKD is the main baseline SemPT is plugged into; the headline results are PromptKD w/SemPT versus PromptKD."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"HPT generates hierarchical LLM descriptions, representing the disparate-description approach SemPT argues is fragmented."},{"cited_title":"Roy and A","cited_arxiv_id":null,"evidence_quote":"CoPrompt enforces consistency with LLM features and is another LLM-augmented baseline in the comparisons."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"ArGue is the attribute-guided prompt-tuning method closest to SemPT; the paper contrasts its own shared vocabulary with ArGue's attribute injection."},{"cited_title":"Zhang, K","cited_arxiv_id":null,"evidence_quote":"CoCoLe distills conceptual knowledge into a codebook, a competing LLM-knowledge representation SemPT compares against."}],"review_version":1}