{"id":"469dd03e-5930-4414-b4d1-90e9b3ef1d57","arxiv_id":"2602.17395","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"SpectralGCD represents images as mixtures over CLIP word-concept similarities, filters the concept dictionary by eigendecomposition of a teacher covariance matrix, and trains a student with forward/reverse distillation to set state-of-the-art accuracy on six GCD benchmarks at lower cost.","lead":"This paper introduces SpectralGCD, a method that represents images by their similarity to a large dictionary of textual concepts and trains a classifier on those scores for Generalized Category Discovery. It reports state-of-the-art accuracy on six benchmarks while training faster than other multimodal methods.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'task-agnostic' dictionary appears to contain the benchmark's novel class names (Table 17), so SpectralGCD may be doing class-name retrieval rather than open-world category discovery; a class-name-stripped dictionary test is needed.","rationale":"The reader's weakest_assumption concerns whether a fixed task-agnostic dictionary plus a frozen CLIP teacher can provide a sufficient representation of class identity, and whether dominant eigenvectors retain class-discriminative semantics. My concern shares the dictionary as the locus of vulnerability but identifies a more specific and more damaging mechanism: the dictionary may contain the actual class names of the novel categories, making the representation sufficient in a trivial and protocol-violating way. The reader did not flag this leakage risk, instead emphasizing threshold tuning on test sets. I agree with the reader's overall CONDITIONAL verdict, but the condition I would impose is different and more central: verify that performance is not driven by the presence of novel class names in the concept vocabulary. If the proposed class-name-stripping test shows a large drop, the paper's claim of discovering novel categories without their names would be substantially weakened, potentially moving the verdict toward REJECT. If the test shows stability, the method's core contribution is strengthened. The manuscript does provide independent support — public code, ablations, dictionary robustness studies — which is why I do not recommend outright rejection without the test.","tokens_in":21320,"tokens_out":7765,"duration_ms":82263,"concrete_test":"Using the released code, construct a 'stripped' dictionary from both Tags and WordNet by removing every concept whose string (after lemmatization, and including multiword entries) matches any Old or New ground-truth class name in each benchmark. Rerun SpectralGCD with the same hyperparameters on CUB, Stanford Cars, FGVC-Aircraft, CIFAR-100, and ImageNet-100, and compare All/Old/New accuracy to Table 1. If fine-grained All accuracy drops by more than ~3 points, the reported state-of-the-art depends on the dictionary containing novel class names, and the claim of agnostic category discovery must be requalified; if accuracy is stable, the leakage concern does not land.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing concern is that the concept dictionary, described as 'task-agnostic', may leak the identities of the novel classes. Section 3 states that GCD uses 'knowledge of Old class names only'; Section 4.1 (Eq. 4) then represents each image by its CLIP similarities to every concept in the dictionary. If that dictionary contains the ground-truth class names of the New classes, then z(x;C) contains direct cues to novel-class identity, and training a classifier on z is closer to semi-supervised class-name retrieval than to discovery.\n\nThis is not hypothetical. Section 5.1 says the default 'Tags' dictionary is 'a list of ~22K concepts arising from several benchmark datasets', and Appendix I (Table 17) reports that Spectral Filtering on CUB with WordNet selects concepts such as 'Tern', 'Warbler', 'Woodpecker', 'Vesper Sparrow', 'Indigo Bunting' — these are exact CUB class names. Neither the Tags nor WordNet dictionary is stripped of benchmark label names, and the manuscript provides no check that the reported gains survive removal of all Old/New class labels. The paper's Limitations section acknowledges dependence on dictionary breadth but does not acknowledge this potential label-name leakage.\n\nIf the dictionary includes the novel class names, the central claim 'state-of-the-art results on six benchmarks while requiring less computation' is not necessarily false, but it would be achieved by exploiting external knowledge of the very categories that GCD assumes are unknown. The spectral filtering and distillation story would then be secondary to the presence of a near-complete label vocabulary in the feature representation.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SpectralGCD, a multimodal generalized category discovery (GCD) method that replaces separate visual and textual streams with a unified cross-modal representation: CLIP image–concept cosine similarities over a large concept dictionary. A spectral filtering stage selects task-relevant concepts using the eigendecomposition of the covariance of teacher softmaxed similarities; training then uses forward and reverse distillation from a frozen teacher plus the standard SimGCD contrastive/parametric losses. Experiments on six benchmarks (CUB, Stanford Cars, FGVC-Aircraft, CIFAR-10/100, ImageNet-100) report state-of-the-art or competitive accuracy at lower training cost than previous multimodal methods, with ablations over thresholds, dictionaries, teachers, student capacity, and Old/New imbalance.","tokens_in":21632,"tokens_out":5960,"duration_ms":55078,"significance":"If the reported results hold, the paper makes a useful practical contribution: it shows that a parametric classifier trained on CLIP cross-modal similarities can match or exceed unimodal and prior multimodal GCD methods at a fraction of the cost, and the public code plus extensive ablations are a strength. The spectral filtering method and distillation combination are well motivated and the efficiency analysis is informative. However, the central claim that the method performs 'category discovery' using only Old class names is threatened by evidence that the concept dictionary contains exact class names of novel categories; this must be resolved before the empirical claim can be interpreted as GCD rather than class-name retrieval.","major_comments":[{"comment":"The dictionary is described as 'task-agnostic' (Abstract, §4.1) and the GCD setup assumes knowledge of Old class names only (§3). However, the default Tags dictionary is 'a list of ~22K concepts arising from several benchmark datasets' (§5.1), and the WordNet run in Table 17 selects concepts such as 'Tern', 'Warbler', 'Woodpecker', 'Vesper Sparrow', and 'Indigo Bunting' — exact CUB class names. Since z(x;C) in Eq. (4) includes similarity to these concepts, a classifier trained on z can identify novel classes by retrieving their names from the dictionary. This converts the task into semi-supervised class-name retrieval and would invalidate the claim of discovering novel categories without their names. I request a systematic check of dictionary overlap with Old/New class names for Tags, OpenImages-v7, and WordNet, and an ablation in which all Old/New class names (and near-synonyms) are rem","section":"§5.1, Appendix I (Table 17)"},{"comment":"The key modeling assumption p(y|x)=p(y|z(x;C)) (Section 4.1) is asserted rather than tested. It is especially important because the dictionary is finite and Table 3 shows that the choice of dictionary changes All accuracy by as much as 3–10 points (e.g., Stanford Cars, OpenImages-v7 vs Tags). If the class-name-stripped experiment from the previous comment shows a large drop, it would indicate that the representation is not sufficient in the intended sense and that the reported gains rely on explicit class-name cues. I would like to see the sufficiency assumption examined by measuring performance against oracle concepts (class names) and against a concept set with all class-name entries removed.","section":"§4.1"}],"minor_comments":[{"comment":"Notation is inconsistent: zθ,ϕ(xi; C) is defined but later written as z(x;C), and the dictionary is sometimes C and sometimes ¯C. Please unify.","section":"Eq. (4) and throughout"},{"comment":"Main-table results are reported as point estimates without standard deviations; deviations appear only in Appendix Table 7. Please include them in the main table or explicitly point the reader to the appendix.","section":"Table 1"},{"comment":"The figure's time comparison would be clearer if the preparation phase for each method were labeled directly on the bars (Spectral Filtering, inversion network, image-to-text assignment), rather than only in the caption.","section":"Figure 4"},{"comment":"The term 'reverse distillation' is nonstandard: L_rd = -Σ σ(z_i) log σ(z*_i) is a cross-entropy of the teacher under the student, not the reverse KL divergence. Please clarify the relationship or rename it (e.g., 'negative cross-entropy').","section":"Eq. (10)"},{"comment":"In the 80% New imbalance setting, TextGCD outperforms SpectralGCD. This is a meaningful boundary on the method's SOTA claim and deserves a sentence in the main paper, not only in the appendix.","section":"Appendix H"}],"recommendation":"major_revision","confidential_remarks":"The label-leakage issue is the main risk. If the class-name-stripped ablation substantially reduces performance, the paper should be reframed as an efficient class-name-retrieval method for GCD benchmarks, not as generalized category discovery. The current evidence (Appendix I) makes this more likely than not. I recommend major revision with the leakage experiment as a prerequisite for acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Couple of things to know about this one. SpectralGCD has the best numbers on six GCD benchmarks, trains a few times faster than GET or TextGCD, and the paper is unusually careful about ablations and reproducibility. The catch: its 'task-agnostic' dictionary is not agnostic. The default Tags dictionary is assembled from benchmark datasets, and Appendix I shows that spectral filtering on CUB with WordNet selects 'Tern', 'Warbler', 'Woodpecker', 'Vesper Sparrow', 'Indigo Bunting' — i.e., exact CUB class names. That means the representation the classifier is trained on contains direct cues to the very novel classes the method is supposed to discover. This is not a hypothetical or a subtle bias; it is the difference between discovering categories and retrieving them from a list that already has the answers.\n\nWhat is genuinely new: training a parametric classifier directly on CLIP image–concept cosine similarities, with eigendecomposition-based concept selection and forward/reverse distillation. The cross-modal representation is a clean idea and the efficiency story holds up: the teacher features are precomputed once and the student is cheap. The experiments are broad — six datasets, three seeds, ablations on dictionary, teacher, student size, Old/New imbalance — and the code link is there. I believe the numbers as reported.\n\nSoft spots, in proportion. The leakage issue is the big one. The paper's Limitations section says the method depends on dictionary breadth, but it never acknowledges that the dictionary contains the benchmark label names. A reviewer should ask for a class-name-stripped dictionary experiment. If the gains vanish, the central claim is substantially weaker. The two threshold parameters beta_e and beta_c are chosen on the same benchmarks that produce the headline results; that is a moderate inflation risk, though the ablation shows sensitivity. The Spearman alignment result is partly forced by the distillation objective, but it is a minor point and the paper does not oversell it.\n\nBottom line: this deserves a serious referee, not a desk reject. The method is interesting, and the leakage is testable. If the results survive a stripped dictionary, it is a strong contribution to GCD. If they do not, the paper still provides a useful efficiency analysis and a cautionary tale about benchmark-derived vocabularies. I would send it to review with a request to add that experiment. For my own work I would wait for the answer before citing it.","headline":"Strong numbers and clean ablations, but the concept dictionary likely leaks the novel class names, so the discovery claim needs a stripped-dictionary test before it convinces.","tokens_in":22165,"tokens_out":3317,"would_cite":false,"duration_ms":31247,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SpectralGCD claims that training a classifier directly on CLIP image-concept similarities, selected by spectral filtering and aligned by forward/reverse distillation, matches or exceeds prior multimodal methods on six category-discovery ben","keywords":["generalized category discovery","cross-modal representation","concept dictionary","spectral filtering","knowledge distillation","CLIP","parametric classification","novel class discovery"],"falsifier":"Take a dataset whose classes differ along concepts missing from a chosen dictionary while sharing strong visual backgrounds; if SpectralGCD's accuracy collapses to near the unimodal baseline, or if the top selected concepts are mostly background words common to all classes, the sufficiency claim is falsified. More directly, one could test conditional independence: whether a classifier on z(x;C) matches one on x as the dictionary is varied.","tokens_in":21181,"feed_emoji":"🧠","tokens_out":5723,"duration_ms":48725,"temperature":0.7,"pith_summary":"Generalized Category Discovery asks a model to cluster unlabeled images into known and novel classes using only a few labels, and this paper's thesis is that the usual recipe of training a classifier on image features overfits to the known classes. SpectralGCD instead trains the classifier on CLIP's image-to-concept similarity vector, expressing each image as a mixture over a large, task-agnostic dictionary of semantic concepts. To keep this representation from drifting, the method first uses spectral filtering on softmaxed teacher similarities to retain only task-relevant concepts, then applies forward and reverse distillation to align the student's representation with the teacher's. The paper reports state-of-the-art accuracy across six benchmarks while requiring less computation than existing multimodal methods and roughly the same as unimodal ones. A sympathetic reader would take this as evidence that cross-modal concept similarities are a sufficient, cheaper representation for category discovery.","feed_headline":"Concept similarities beat image+text pipelines in category discovery","feed_subtitle":"Spectral filtering of a CLIP concept space cuts training cost while matching or beating multimodal rivals.","key_machinery":"The central object is the cross-modal representation z_{theta,phi}(x;C), a vector of CLIP cosine similarities between an image and every concept in a task-agnostic dictionary, treated as a mixture over semantics rather than as separate visual and textual features. Two mechanisms carry the argument. Spectral Filtering forms a cross-modal covariance matrix on softmax-normalized teacher similarities, keeps the eigenspace that explains a threshold of variance, and computes a concept importance score s_j = sum_i lambda_i v_{ij}^2 to select the dictionary subset that retains class-discriminative concepts. Forward and reverse knowledge distillation losses then transfer the teacher's concept-distrib","core_discovery":"SpectralGCD's central claim is that class identity can be read off a cross-modal representation z(x;C), the vector of cosine similarities between an image and concept names from a large agnostic dictionary, so that p(y|x) is well approximated by p(y|z(x;C)). Because the dictionary is noisy, Spectral Filtering computes the covariance of softmaxed teacher similarities over the dataset, takes its top eigenvectors, and sums squared eigenvector components weighted by eigenvalues to score and retain only concepts whose co-activation carries signal. The retained concepts define the student's representation, and forward plus reverse distillation from the same frozen teacher keeps the student's simil","pith_inferences":["If the sufficiency assumption p(y|x)=p(y|z(x;C)) holds approximately, then the bottleneck shifts from the classifier to the dictionary's concept coverage; a natural extension is to make the concept set image-adaptive rather than fixed.","The same spectral-filtering recipe may transfer to other label-scarce settings, such as open-set recognition or continual discovery, whenever a frozen vision-language teacher and a large concept list are available.","Covariance-based selection is an unsupervised proxy for class discrimination; a direct test would compare selected concepts against a supervised importance ranking and check whether the two agree on fine-grained datasets.","The reported robustness to generic linguistic dictionaries suggests the method could scale to 100k+ concepts with low-rank covariance approximations, trading a small accuracy dip for much broader coverage."],"forward_implications":["A classifier trained on filtered concept similarities should keep performing on novel classes, since it is not anchored to spurious visual cues that only separate known classes.","Because the text encoder is frozen and teacher similarities are precomputed once, the training overhead stays close to unimodal levels even though the method is multimodal.","The discovery pipeline can be re-run cheaply as new unlabeled data arrive, which the paper argues is the realistic deployment setting.","On several benchmarks the distilled student surpasses the zero-shot accuracy of its own teacher, suggesting the method extracts more from the concept space than the teacher does at inference time.","Using an unfiltered dictionary hurts performance on fine-grained datasets, so concept selection is a direct accuracy lever, not just a computational one."],"fun_headline_variants":["Spectral filtering trims concept space for faster category discovery","CLIP concept similarities alone outperform image-text multimodal stacks","Cross-modal concept vectors match multimodal GCD at a fraction of cost","Spectral concept selection slashes training cost without accuracy loss","One concept similarity vector replaces full image-text fusion for GCD"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The fixed task-agnostic concept dictionary plus the frozen teacher's similarity structure must actually capture class identity; if the dictionary lacks the discriminating concepts, or the leading covariance eigenvectors are driven by dataset-specific background bias rather than class semantics, the representation loses the information the classifier needs.","fun_headline_variants_meta":{"raw":{"variants":["Spectral filtering trims concept space for faster category discovery","CLIP concept similarities alone outperform image-text multimodal stacks","Cross-modal concept vectors match multimodal GCD at a fraction of cost","Spectral concept selection slashes training cost without accuracy loss","One concept similarity vector replaces full image-text fusion for GCD"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000387,"raw_usage":{"total_tokens":1876,"prompt_tokens":736,"completion_tokens":1140,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":480,"completion_tokens_details":{"reasoning_tokens":1059}},"tokens_in":480,"tokens_out":1140,"duration_ms":8509,"temperature":1.0,"reasoning_tokens":1059,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T22:13:45.589589+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a dataset whose classes differ along concepts missing from a chosen dictionary while sharing strong visual backgrounds; if SpectralGCD's accuracy collapses to near the unimodal baseline, or if the top selected concepts are mostly background words common to all classes, the sufficiency claim is falsified. More directly, one could test conditional independence: whether a classifier on z(x;C) matches one on x as the dictionary is varied.","supporting_citations":[],"review_version":1}