{"id":"b2769b3e-4b3a-4368-9f24-f6e9c5b427c0","arxiv_id":"2505.03361","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"InfZSL generates unlimited LLM-based visual concepts for each class, selects transferable and discriminative ones via a concept-entropy ranking, and builds interpretable class embeddings that improve zero-shot recognition on three benchmarks.","lead":"The paper proposes InfZSL, a zero-shot learning method that uses large language models to generate many visual concept phrases per class, then automatically filters and scores them with a new entropy-based measure of transferability and discriminability. The authors report small accuracy gains over recent baselines on AWA2, CUB, and SUN, along with attention maps suggesting the selected concepts are image-grounded.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"LLM co-occurrence counts are used as visual grounding without validation; the 'image-grounded' claim rests on an untested proxy.","rationale":"The paper's central claim has two components: accuracy improvements and interpretable, image-grounded concepts. The latter is the novel contribution; if the co-occurrence proxy fails, the method's main differentiator over document-based ZSL disappears. The accuracy gains are small and could partly come from the I2CFormer architecture, but the interpretability claim stands or falls on the proxy. The proposed test is feasible and would settle the issue. This concern matches the reader's weakest assumption, so the conditional verdict is retained without change.","tokens_in":15082,"tokens_out":5942,"duration_ms":59177,"concrete_test":"Take 50 AWA2 classes and 50 randomly selected concept clusters that pass CESS. For each class, download 20 test images; for each (class, concept) pair, compute the CLIP similarity between the image and the concept phrase (e.g., 'a photo of a {concept}'), averaged over images. Then compute the Spearman correlation between this visual-grounding score and the co-occurrence count o_{i,j} used in Section 3.3. If the correlation is not significantly positive (e.g., rho < 0.3), the proxy fails and the 'image-grounded' claim is unsupported. As a secondary check, compare CESS-selected concepts against randomly chosen concepts on the same visual-grounding score; they should be clearly higher.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.3 builds the entire CESS selection on the class-cluster co-occurrence matrix o_{i,j}, which counts how often each LLM-generated concept cluster appears in the LLM's concept list for each class (Eqs. 1-4). Section 3.4 then scores the selected concepts by the same co-occurrence counts to form the class embedding S_m. No image data enters anywhere in this pipeline. The paper's central claim that selected concepts are 'image-grounded' therefore depends on the assumption that LLM co-occurrence frequency is a faithful proxy for visual presence in images of that class. This assumption is not checked in the main paper; the only validation is a user study (Appendix A.5) covering 4 classes, 10 concepts, and 29 volunteers. A hallucinated but class-correlated phrase (e.g., 'nocturnal hunting' or 'mythological symbolism' for a subset of predators) would have moderate entropy and moderate discriminability and could pass the harmonic-mean filter despite being absent from images. If this happens, the embedding dimensions are not visually grounded, which undermines both the interpretability claim and the transfer to unseen classes. The concern is compounded by dataset-specific hyperparameters (kpre, kselect, ktop) chosen without a validation protocol, but the proxy assumption is the load-bearing issue.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes InfZSL, a zero-shot learning framework that replaces human-annotated attribute vectors with LLM-generated phrase concepts. It generates many concepts per class via a prompt template, clusters them with GloVe embeddings and k-means, computes a class-cluster co-occurrence matrix O, and defines two scores derived from O: a 'concept entropy' transferability score and a ktop-order-probability discriminability score. The harmonic mean of the two ranks the clusters, the top kselect clusters are used to build a class embedding S_m, and S_m is concatenated with human-annotated attributes and fed into an embedding method (I2CFormer) or a generative method (ZeroDiff). Experiments on AWA2, CUB, and SUN report consistent improvements over several concept-based and document-based ZSL baselines, with ablations indicating that each component contributes.","tokens_in":15319,"tokens_out":7441,"duration_ms":69790,"significance":"If the results hold, InfZSL would be a useful step toward reducing annotation cost while retaining interpretability in ZSL. The paper's strengths are a clean three-stage pipeline, comparisons against current concept-based and document-based baselines on three standard benchmarks, component ablations showing the contributions of transferability and discriminability selection, and the use of GloVe rather than CLIP to avoid leaking unseen-class visual information. The improvements over the generative baseline ZeroDiff, however, are small, and the central 'image-grounded' claim currently rests on a single small user study and on the unvalidated assumption that LLM co-occurrence counts proxy visual presence.","major_comments":[{"comment":"The co-occurrence matrix O is built by counting how often each LLM-generated concept cluster appears in the LLM's concept lists for each class, so both the selection scores and the final embedding S_m are computed from the same LLM text counts and no image information enters the pipeline. The paper's claim that selected concepts are 'image-grounded' is therefore unsupported; the only direct validation, the user study in Appendix A.5, covers 4 classes, 10 concepts, and 29 volunteers, which is too small to bear the weight of the claim. Please add a larger image-based validation (e.g., human ratings of concept presence in images across many classes, or an automated visual check that does not leak unseen-class data) or substantially weaken the 'image-grounded' wording.","section":"Section 3.3, Eqs. (1)-(4); Section 3.4"},{"comment":"The definition of o_{i,j} is underspecified: the text calls it a 'class-cluster co-occurrence percent' but does not state the exact normalization (per class, per cluster, across the ktime sampled concept lists, after or before deduplication of concepts within a class). Because Eq. (1) applies a softmax to o directly, the chosen normalization changes p(y|q) and hence both r_tran and r_dis, so the reported selection results cannot be reproduced without this information.","section":"Section 3.3, Eq. (1)"},{"comment":"The concept importance r_q is defined as the harmonic mean of a transferability score that is normalized across clusters and a discriminability score that is an order statistic of the softmax probabilities and is not normalized across clusters. These quantities have different scales, and the harmonic mean is introduced without derivation or comparison to alternative fusion functions (e.g., product, weighted sum, min). Add an ablation of the fusion function or a theoretical justification, and report whether the selected concept sets and final accuracies are sensitive to this choice.","section":"Section 3.3.3"},{"comment":"The gains over the strongest baselines are small in several cases (e.g., InfZSL+ZeroDiff over ZeroDiff is +0.2 H on AWA2 and +0.7 H on CUB), and no error bars, multiple seeds, or significance tests are reported, so the claimed 'significant improvements' are not established. Similarly, the hyperparameters (kpre, kselect, ktop) are chosen empirically per dataset without a validation protocol, and the sensitivity analysis in Fig. 4 covers only AWA2 and CUB, not SUN. Report run-to-run variance and a validation-based selection procedure for hyperparameters.","section":"Section 4, Table 1"}],"minor_comments":[{"comment":"The phrase 'resulting our our concept-based semantic embeddings' contains a duplicated 'our'; also correct 'intergrated' in Section 4 and 'promt' in Section 3.2.","section":"Section 3.4"},{"comment":"The axis label 'AUSUC' is never defined in the text; please define it or rename the metric.","section":"Figure 4"},{"comment":"The text says InfZSL achieves 'best performance 76.5%' on AWA2 ZSL, but Table 1 reports 76.6; please correct the inconsistency.","section":"Section 4.1"},{"comment":"The notation '`p' for the sorted probability sequence is not defined; please define it explicitly.","section":"Eq. (4)"},{"comment":"The caption refers to 'I2DFormer' in the architecture description, but the model in the figure and text is I2CFormer.","section":"Figure 12"},{"comment":"The statement that 'Inf.'+'Tran.' generally have large improvements overstates the numbers in Table 2; the differences are 0.7-2.7 points and not consistently 'large'.","section":"Section 4.2.1"},{"comment":"The claim that the method 'sets a new standard for interpretable ZSL' is too strong given the limited grounding validation.","section":"Section 1"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a timely problem and the initial results are promising, but the 'image-grounded' claim is central to the contribution and currently rests on very limited evidence. The definitional gaps in the co-occurrence matrix and the lack of variance reporting further weaken the paper. I would encourage the editor to require the additional validation and reproducibility details before considering acceptance. The 'infinite' in the title is also somewhat misleading, as the method generates a finite number of concepts per class."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nRead this one because it claims to solve LLM hallucination for interpretable zero-shot learning. Quick take: the pipeline is novel and the experiments are honestly reported, but the 'image-grounded' claim is not supported by the evidence.\n\nThe new thing here is CESS: generate a large set of phrase-level concepts from an LLM, cluster them by GloVe embeddings, rank clusters by the harmonic mean of a transferability score (concept entropy over class-conditioned co-occurrence probabilities) and a discriminability score (the k-th largest class probability), then build the class semantic embedding by counting co-occurrences of selected clusters per class. That combination—infinite generation, entropy-based selection for both transferability and discriminability, and co-occurrence scoring—is, as far as I can tell, new. It sensibly targets a real problem: LLM-generated documents for ZSL are opaque and contain non-visual filler.\n\nWhat's good: the ablations in Table 2 show each component contributes, and the gains over the strongest generative baseline (ZeroDiff) are consistent across AWA2, CUB, and SUN, even if most are under one point. The qualitative heatmaps and attention maps are plausible. The writing is clear.\n\nThe soft spot is load-bearing. Section 3.3 builds the entire selection and scoring on the class-cluster co-occurrence matrix o_{i,j}, which is simply how often the LLM attaches a concept to a class name in the prompt-generated lists. No image data enters anywhere. The paper asserts that this filters hallucinated concepts, but it never checks that LLM co-occurrence frequency is a proxy for visual presence. The user study in Appendix A.5 covers 4 classes, 10 concepts, and 29 volunteers—too small to establish that. If a concept like 'cunning behavior' appears consistently for one class, it will look transferable and discriminative by the metric, yet it is not visual. That undermines the interpretability claim, though not necessarily the accuracy numbers.\n\nAlso minor: no code or LLM configuration is released; hyperparameters (kpre, kselect, ktop) are chosen per dataset without a validation protocol; no error bars or significance tests; the harmonic mean is introduced without derivation. But these are fixable.\n\nWho this is for: people working on LLM-based ZSL or concept bottleneck models. It is a solid incremental contribution, worth a serious referee. If I were editor, I'd send it out, but with a request for code, a validation of the co-occurrence proxy against images (or at least a larger user study), and error bars.\n\nRegards.","headline":"A coherent pipeline for LLM-generated concept-based ZSL with consistent but modest gains; the 'image-grounded' claim outruns the evidence.","tokens_in":15855,"tokens_out":3206,"would_cite":true,"duration_ms":30153,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that LLM-generated phrase concepts, filtered by an entropy score, can replace finite human-attribute annotations in zero-shot learning while making the class embedding interpretable.","keywords":["zero-shot learning","class concepts","large language models","hallucination","concept entropy","interpretability","semantic embedding"],"falsifier":"A concrete failing observation would be a selected concept that is consistently invisible in the class's images, for example 'cunning gaze' for fox showing no stable visual correlate in a ground-truth attribute set. An experiment could take the ten concepts from the paper's user study, annotate their presence in 50 randomly sampled images per class, and check whether the CESS-selected concepts are genuinely more visually present than the eliminated ones; near-chance separation would refute the image-grounded claim.","tokens_in":14865,"feed_emoji":"🧠","tokens_out":4564,"duration_ms":41203,"temperature":0.7,"pith_summary":"The paper tries to show that zero-shot learning need not depend on expensive, finite human-attribute annotations: a large language model can generate any number of phrase-level visual concepts, and an entropy-based filter can keep only those that are both transferable across classes and discriminative between them. InfZSL builds the class semantic embedding from co-occurrence counts of the surviving concepts, then concatenates it with the human-annotated embedding and plugs into existing embedding or generative zero-shot methods. A sympathetic reader would care because, if true, each dimension of the class embedding becomes a human-readable phrase like 'long neck' rather than an opaque document vector, and the method reports consistent improvements over its host backbones on AWA2, CUB, and SUN without using unseen-class image supervision.","feed_headline":"LLM concepts beat human attributes in zero-shot learning","feed_subtitle":"A concept-entropy filter keeps only transferable, discriminative, image-grounded phrases, giving interpretable class embeddings.","key_machinery":"The load-bearing object is the Concept Entropy Selection and Scoring (CESS) procedure. CESS first clusters LLM-generated concepts using GloVe embeddings and k-means, then for each cluster computes a class-cluster co-occurrence matrix, derives a concept-entropy transferability score and a ktop-th-probability discriminability score, and ranks clusters by the harmonic mean of the two scores. Selected clusters are scored by class-concept co-occurrence counts to form the LLM-concept semantic embedding, which is concatenated with the human-annotated embedding before training the zero-shot classifier.","core_discovery":"The central claim is that an LLM can supply an effectively infinite set of phrase-level class concepts that, once filtered and scored by the Concept Entropy Selection and Scoring (CESS) procedure, produce a semantic embedding that is at least as accurate as human-annotated attribute embeddings and far more transparent. The paper argues that hallucinated or non-visual phrases like 'cunning behavior' can be detected at the individual-concept level by concept entropy, which measures how evenly a phrase's co-occurrences spread over classes, and that this signal is complementary to a discriminability score based on the ktop-th class probability. Combining the two in a harmonic mean lets the method keep only concepts like 'long neck' that are both shared across classes and informative within them, and the surviving concepts form the class embedding through class-concept co-occurrence counts.","pith_inferences":["If CESS truly filters hallucinated concepts using only text statistics, the same proxy could plausibly be applied to other vision-language problems such as concept bottleneck models or explainable retrieval, though the paper does not test those settings.","A direct extension would be to run CESS on classes that already have visual attribute ground truth, such as CUB, and check whether eliminated concepts are systematically less visually present than selected ones; the paper's own user study covers only four classes and ten concepts.","Because the method concatenates LLM and human embeddings, some of the reported gain could come from added embedding dimensions rather than from the filtering itself; a dimension-matched ablation would isolate the contribution of CESS.","The 'image-grounded' claim is testable only against pixel-level evidence, so a larger human study or an automatic visual-grounding metric would be the natural next check."],"forward_implications":["InfZSL can be integrated into both embedding and generative zero-shot backbones; the paper reports that it improves its host methods on all three benchmarks, with the largest gains on AWA2.","The final class embedding is interpretable: every dimension corresponds to a phrase such as 'small dorsal fin' or 'long vibrissae', and the paper shows attention maps that localize these phrases in images.","The transferability-discriminability tradeoff suggests a general recipe for mining visual semantics from LLMs: filter by a harmonic-mean importance score rather than by raw frequency or by a single fidelity score.","Because the pipeline needs only class names, LLM prompts, and GloVe embeddings, it could scale to datasets where human attribute annotation is impractical or too costly."],"supporting_citations":[{"why":"Supplies the GloVe word embeddings used to represent and cluster LLM-generated concepts.","marker":"[24]"},{"why":"Provides the semantic-entropy hallucination detection idea that the paper adapts from whole-response level to individual-concept level.","marker":"[11]"},{"why":"Contributes the LLM-generated concept paradigm and the concept scoring idea that InfZSL extends with transferability and discriminability selection.","marker":"[41]"},{"why":"Supplies the multi-document LLM supervision baseline and the prompt template that InfZSL adapts for concept generation.","marker":"[20]"},{"why":"The document-based zero-shot learning baseline whose image-to-document attention framework inspires the I2CFormer embedding backbone.","marker":"[21]"},{"why":"Documents the LLM hallucination problem that motivates the concept filtering stage.","marker":"[14]"},{"why":"The generative zero-shot baseline that InfZSL integrates with and improves upon.","marker":"[42]"},{"why":"Supplies the CUB benchmark and the class-concept co-occurrence scoring convention used to build semantic embeddings.","marker":"[34]"}],"fun_headline_variants":["LLM infinite concepts, entropy-filtered, for interpretable ZSL","Infinite LLM concepts beat human attributes in zero-shot","Zero-shot with infinite interpretable concepts from LLMs","Concept entropy filters LLM hallucinations for grounded ZSL","Unlimited LLM phrases, selected for transferability, in ZSL"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire pipeline assumes that how often an LLM attaches a phrase to a class name is a faithful proxy for whether that phrase is visually present in images of the class; if an LLM hallucinates a class-specific but non-visual phrase, it will look discriminative and pass the entropy filter.","fun_headline_variants_meta":{"raw":{"variants":["LLM infinite concepts, entropy-filtered, for interpretable ZSL","Infinite LLM concepts beat human attributes in zero-shot","Zero-shot with infinite interpretable concepts from LLMs","Concept entropy filters LLM hallucinations for grounded ZSL","Unlimited LLM phrases, selected for transferability, in ZSL"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000786,"raw_usage":{"total_tokens":3439,"prompt_tokens":889,"completion_tokens":2550,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":505,"completion_tokens_details":{"reasoning_tokens":2465}},"tokens_in":505,"tokens_out":2550,"duration_ms":18048,"temperature":1.0,"reasoning_tokens":2465,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T23:53:04.316374+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A concrete failing observation would be a selected concept that is consistently invisible in the class's images, for example 'cunning gaze' for fox showing no stable visual correlate in a ground-truth attribute set. An experiment could take the ten concepts from the paper's user study, annotate their presence in 50 randomly sampled images per class, and check whether the CESS-selected concepts are genuinely more visually present than the eliminated ones; near-chance separation would refute the image-grounded claim.","supporting_citations":[{"cited_title":"I2mvformer: Large language model generated multi-view document supervision for zero-shot image classification","cited_arxiv_id":null,"evidence_quote":"Supplies the multi-document LLM supervision baseline and the prompt template that InfZSL adapts for concept generation."},{"cited_title":"I2dformer: Learning image to document attention for zero-shot image classification","cited_arxiv_id":null,"evidence_quote":"The document-based zero-shot learning baseline whose image-to-document attention framework inspires the I2CFormer embedding backbone."},{"cited_title":"The caltech-ucsd birds-200-2011 dataset","cited_arxiv_id":null,"evidence_quote":"Supplies the CUB benchmark and the class-concept co-occurrence scoring convention used to build semantic embeddings."}],"review_version":1}