{"id":"9f635f7c-1ea1-458a-b11a-4908fe7ecb2a","arxiv_id":"2411.12584","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"TRIDENT improves compositional zero-shot recognition by using LLaVA hidden states as word embeddings and smoothing attribute labels with auxiliary adjectives generated by GPT-3.5.","lead":"This paper introduces TRIDENT, a compositional zero-shot learning model that uses multimodal LLM embeddings and LLM-generated attribute smoothing to recognize unseen attribute-object pairs. It reports state-of-the-art accuracy on MIT-States, C-GQA, and VAW-CZSL.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 2 omits the current VLM-based CZSL methods (DSP, Troika) that the paper itself cites; the \"state-of-the-art\" claim in §4.2 is therefore unverified, and their published results are sufficient to overturn it if added.","rationale":"The reader's verdict is CONDITIONAL and centers on potential LLaVA leakage. I agree that leakage is a real risk, but the more immediately checkable load-bearing problem is the omitted SOTA comparison: the paper cites DSP and Troika as recent VLM-based CZSL methods yet never includes them in Table 2, so the headline claim cannot be verified from the paper. This is a concrete, falsifiable omission rather than a speculative failure mode. The method itself has internal support: ablations show each component helps, the word-embedding comparison is informative, and the auxiliary-attribute quality checks are a good faith effort. Those strengths do not, however, establish SOTA. The verification step is straightforward: add the missing rows; if the omitted methods are lower, the claim survives and the reader's conditional concerns (code, variance, leakage) remain the remaining conditions. If the omitted methods are higher, the central claim must be downgraded. Either way, the current manuscript lacks the evidence needed to certify the abstract's strongest claim, so keeping the CONDITIONAL verdict is appropriate.","tokens_in":17904,"tokens_out":14566,"duration_ms":149473,"concrete_test":"Insert rows for DSP (Lu et al., CVPR 2023) and Troika (Huang et al., CVPR 2024) into Table 2 using their originally published results, and also rerun them under the authors' ViT-L/14-336 retraining protocol. If either method's AUC or HM exceeds TRIDENT's 14.2/30.9, 8.0/22.6, or 8.3/23.4 on MIT-States, C-GQA, or VAW-CZSL, revise the abstract and §4.2 to say \"competitive\" rather than \"state-of-the-art.\"","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.1 lists baselines as SymNet, CompCos, Co-CGE, SCEN, OADis, INV, CANet, ProCC, plus CLIP and CoOp, but does not include DSP [Lu et al., CVPR 2023] or Troika [Huang et al., CVPR 2024], both cited in Related Work as VLM-based CZSL methods. Section 4.2 then claims \"new state-of-the-art performance\" solely by comparison with CANet. On the same three benchmarks, the omitted methods report results that are at least competitive with, and in several cases higher than, TRIDENT's 14.2/30.9, 8.0/22.6, and 8.3/23.4 AUC/HM. The paper's fairness caveat about CLIP in §4.1 does not justify this omission, since CLIP and CoOp are themselves included as rows and since TRIDENT's own visual backbone is LLaVA's CLIP ViT-L/14. Consequently, the central SOTA claim is not falsifiable from the paper as written. This concern is independent of the Appendix C leakage question: even granting that LLaVA has not memorized test labels, the comparison set is too narrow to support the unqualified \"state-of-the-art\" statement in the abstract and §4.2.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes TRIDENT, a compositional zero-shot learning (CZSL) framework built on a frozen LLaVA v1.5 visual encoder. The method extracts multi-granularity visual features through feature adaptive aggregation modules and learnable condition masks, disentangles attributes and objects using weighted shared/exclusive features from image triplets, and aligns visual features with MLLM last-hidden-state word embeddings. It additionally introduces attribute smoothing, where GPT-3.5-generated auxiliary attributes soften the attribute classification target. Experiments on MIT-States, C-GQA, and VAW-CZSL report improved AUC and harmonic-mean accuracy over the included baselines, and the paper claims state-of-the-art performance.","tokens_in":18143,"tokens_out":11468,"duration_ms":105739,"significance":"If the experimental comparison is complete and reproducible, the method would be a useful empirical contribution to CZSL. The paper deserves credit for a systematic ablation study, hyperparameter sensitivity analysis, and an auxiliary-attribute quality assessment, and for retraining the non-CLIP baselines with a common backbone. However, the significance currently rests on an unqualified state-of-the-art claim that is not supported by the comparison set as presented: two VLM-based CZSL methods cited in the paper itself, DSP and Troika, are absent from Table 2. In addition, the use of LLaVA v1.5 as both the visual backbone and the source of word embeddings raises a data-contamination risk that Appendix C addresses only by assertion. These issues are load-bearing for the paper's central claim.","major_comments":[{"comment":"The unqualified \"state-of-the-art\" claim is not supported by the comparison set. The paper cites DSP (Lu et al., CVPR 2023) and Troika (Huang et al., CVPR 2024) as VLM-based CZSL methods in Related Work, but neither appears in Table 2, and Section 4.2 declares CANet the previous state of the art. Published results of DSP and Troika on the same three benchmarks are at least competitive with and in several cases higher than TRIDENT's reported AUC of 14.2, 8.0, and 8.3, so the sentence \"TRIDENT boosts AUC from ... CANet ... to new state-of-the-art\" is not verifiable as written. Including CLIP and CoOp rows does not cure this, and the fairness caveat about CLIP in Section 4.1 is not a justification for omitting two directly comparable VLM-based methods. Please extend Table 2 with DSP, Troika, and any other current CZSL methods, or remove the unqualified SOTA statement until such a comparison is performed.","section":"§4.1–4.2, Table 2"},{"comment":"The zero-shot interpretation of the results depends on a premise that Appendix C states but does not support. The appendix concedes that \"LLaVA v1.5 may have seen certain images during training\", then asserts that the autoregressive captioning objective makes the model focus on main content rather than the attribute-object label. That assertion is not evidence: LLaVA v1.5's visual encoder is a CLIP ViT-L/14 trained on large-scale web image-text data, and its instruction-tuning data contains captioned images, so near-duplicate test images or attribute-object phrases could plausibly have been memorized. Because TRIDENT uses LLaVA-derived visual features and LLaVA-derived word embeddings, any such memorization would directly inflate unseen accuracy. Please add an empirical contamination check, such as LLaVA-based retrieval or zero-shot naming on the exact test splits, or a comparison with a backbone whose pretraining data is known to be disjoint; if such a check is infeasible, the paper should explicitly present this as a limitation and temper the generalization claims.","section":"Appendix C; §3.2"},{"comment":"The headline improvements are reported from a single run without variance or significance estimates. On MIT-States the gain over CANet is 0.6 AUC absolute (14.2 vs 13.6), and several ablation deltas in Table 3 are of the same magnitude, so run-to-run variation could easily change the conclusion. Please report mean and standard deviation over at least three random seeds and indicate whether the TRIDENT-versus-CANet differences are significant; otherwise, the phrase \"substantial margin\" in Section 4.2 overstates the evidence.","section":"Table 2; §4.2"},{"comment":"The paper is internally inconsistent about whether the MLLM embeddings are frozen. Section 3.2 says \"The auxiliary attributes and MLLM embeddings are obtained offline before training TRIDENT\"; Appendix C says \"we fine-tune the last hidden states with a low learning rate during the training phase\"; and Section 3.3 lists a separate learning rate of 1.5e-6 for word embedding. Please state unambiguously which parameters are updated during training (LLaVA transformer, cross-modal connector, the projection MLP, or only a final linear layer) and clarify whether the word embeddings used in Eq. (9) and Eq. (10) change during training. This is essential for reproducibility and for interpreting Table 4, where \"LLaVAlhs\" may denote either frozen or fine-tuned embeddings.","section":"§3.2; Appendix C; §3.3"}],"minor_comments":[{"comment":"Equation (10) applies a linear layer directly to LLaVA_lhs(a) and LLaVA_lhs(o), whereas Eq. (9) defines the word embedding as E_word(y) = MLP_word(LLaVA_lhs(y)); please make the composition embedding use E_word(a) and E_word(o) or explain why the MLP is skipped in Eq. (10).","section":"Eq. (9)–(10)"},{"comment":"Table 1 and Appendix D disagree on VAW-CZSL: Table 1 lists 1,252 seen training compositions, while Appendix D states 11,175 seen compositions; please reconcile the numbers and report the exact split used.","section":"Table 1; Appendix D"},{"comment":"It is stated that all baselines were retrained with a common ViT-Large-Patch14-336px backbone for fairness, but the CLIP and CoOp rows appear to be taken from a prior paper rather than retrained under the same protocol; please state explicitly which entries were re-run and which were copied.","section":"§4.1, Table 2"},{"comment":"The hyperparameter sensitivity curves for the number of visual features and the label smoothing factor do not state whether they are computed on the validation split; if the hyperparameters were selected using the test split, this should be disclosed.","section":"Appendix E"}],"recommendation":"major_revision","confidential_remarks":"The main blocker is the omitted comparison set: DSP and Troika are cited in Related Work but absent from Table 2, and their published numbers are sufficient to challenge the SOTA claim. This is fixable within the scope of the paper by adding the baselines and revising the claim accordingly, so I do not recommend rejection. The LLaVA data-contamination issue also needs an explicit response, preferably with a concrete empirical check, before the generalization claim can be taken at face value."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The short version: this is a credible CZSL paper that combines MLLM last-hidden-state embeddings with LLM-generated attribute smoothing. The ablations show both contribute, the engineering is competent, and the paper is honest about its own uncertainties. But the headline 'state-of-the-art' claim is not supported as written, because the comparison table omits two strong VLM-based baselines that the authors themselves cite in Related Work.\n\nWhat is new: using LLaVA's last hidden states as word embeddings for attributes and objects, and smoothing attributes toward LLM-generated auxiliary words. Both are borrowed from outside CZSL (SGPT/GritLM for embeddings, label smoothing for the latter), but they are new in this context, and Table 3 suggests each helps. The FAA modules and learnable condition masks are incremental but reasonable. The ablation study is the paper's strongest part: each component is removed, and hyperparameter sensitivity is shown in the appendix.\n\nThe first soft spot is the baseline selection. Section 4.1 lists SymNet, CompCos, Co-CGE, SCEN, OADis, INV, CANet, ProCC, CLIP, CoOp—but not DSP (Lu et al., CVPR 2023) or Troika (Huang et al., CVPR 2024), both cited as VLM-based CZSL methods. Section 4.2 then claims 'new state-of-the-art performance' by comparison to CANet. That claim cannot be verified from the paper. The published numbers for those two methods are competitive with, and in some cases higher than, TRIDENT's. The paper's excuse that CLIP-based comparisons are 'unfair' does not hold, because TRIDENT's visual backbone is LLaVA's CLIP ViT-L/14; the same exposure concern applies to TRIDENT itself.\n\nSecond, the leakage question is real. Appendix C says LLaVA 'may have seen certain images' but that its generative training focuses on descriptions, not attribute-object labels. That is not a satisfying argument. The visual encoder could have memorized test images or labels from web captions. The paper acknowledges the issue, but does not measure it. This is not an internal circularity—the losses are standard cross-entropy on seen data and evaluation is on held-out compositions—but it is a benchmark-validity risk.\n\nMinor items: single runs, no variance; the MIT-States gain is 0.6 AUC, within noise. Code and auxiliary attributes are promised but not released at review time.\n\nWho this is for: researchers who work on CZSL and want to see whether MLLM embeddings plus LLM-generated smoothing help beyond CLIP-only methods. It deserves a serious referee: the method is coherent, the ablations are honest, and the missing baselines can be added.\n\nRecommendation: send to peer review, but require the authors to add DSP and Troika to the comparison (at minimum by citing their reported numbers), report multiple runs, release code and generated attributes, and treat the leakage concern as an open problem rather than dismissing it in a paragraph.","headline":"A credible CZSL engineering paper with a useful trick and an honest ablation study, but the state-of-the-art claim is not supported because the comparison omits two strong VLM baselines the authors themselves cite.","tokens_in":18741,"tokens_out":5596,"would_cite":false,"duration_ms":48597,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"TRIDENT, a disentanglement framework with MLLM embeddings and attribute smoothing, sets new state-of-the-art results on MIT-States, C-GQA, and VAW-CZSL.","keywords":["compositional zero-shot learning","attribute-object disentanglement","multimodal large language models","MLLM embeddings","attribute smoothing","label smoothing","visual disentanglement","generalized zero-shot learning"],"falsifier":"Run the identical TRIDENT pipeline with a frozen visual encoder trained only on ImageNet-1k, with no web-scale caption data, and compare the AUC on the three benchmarks; if the margin over CANet shrinks to near zero while the rest of the architecture is unchanged, the reported gains are attributable to pretraining-data overlap with the test splits rather than to disentanglement and attribute smoothing.","tokens_in":17635,"feed_emoji":"🧩","tokens_out":10249,"duration_ms":83455,"temperature":0.7,"pith_summary":"Compositional zero-shot learning asks a model to recognize never-seen pairings of attributes and objects, such as a peeled orange, from training on seen pairings such as ripe orange and peeled apple. The paper argues that previous disentanglement methods fail for three reasons: background pixels contaminate the visual features used for separation, static word vectors like GloVe carry no multimodal semantics, and one-hot attribute labels make models overconfident about seen compositions. To fix this, the paper introduces TRIDENT, which filters background with feature adaptive aggregation, splits global features with learnable condition masks, backs alignment with the last hidden states of a multimodal LLM, and softens attribute labels using auxiliary attributes generated by a text LLM. The claim is that TRIDENT is the new state of the art on MIT-States, C-GQA, and VAW-CZSL, with the largest gains on unseen compositions. If true, it strengthens the case that large pretrained multimodal models plus soft semantic labels, rather than task-specific architectural tricks alone, drive compositional generalization.","feed_headline":"TRIDENT lifts CZSL AUC to 14.2, 8.0, 8.3 on three benchmarks","feed_subtitle":"LLaVA hidden-state embeddings plus LLM attribute smoothing beat CANet on all three CZSL benchmarks.","key_machinery":"The load-bearing mechanism is a triplet-based weighted disentanglement module coupled to a softened semantic alignment. From a main image and two partner images sharing either the attribute or the object, two MLPs emit per-feature shared and exclusive weights, and the exclusive weights are computed as $1 - w$ so that each row of the visual feature matrix is split into attribute and object components rather than relying on spatial region separation. Visual extraction is itself split: $p$ feature adaptive aggregation modules use $1\\times1$ convolution to weight patch tokens and suppress background, while $q$ learnable condition masks multiply the [CLS] token elementwise to spread global information across dimensions. For alignment, every attribute, object, and auxiliary word is mapped through the average-pooled last hidden states of LLaVA v1.5, its MLLM embeddings, and the cosine cross-entropy treats all non-ground-truth words as negatives. Finally, attribute smoothing replaces the one-hot attribute target with a distribution that puts weight $1-\\alpha$ on the ground-truth attribute and $\\alpha/t$ on $t$ auxiliary attributes generated by GPT-3.5, so the model is not trained to reject all other plausible attributes.","core_discovery":"The paper's central discovery is that replacing static word embeddings with the last hidden states of an MLLM and replacing one-hot attribute targets with smoothed targets derived from LLM-generated auxiliary attributes materially improves compositional zero-shot recognition. Concretely, TRIDENT reports area-under-curve of 14.2% on MIT-States, 8.0% on C-GQA, and 8.3% on VAW-CZSL, compared with 13.6%, 5.7%, and 6.7% for the previous best method CANet, and harmonic-mean improvements from 29.8%, 18.9%, and 21.0% to 30.9%, 22.6%, and 23.4%. The authors attribute the gains to three mechanisms working together: background suppression and multi-granularity feature splitting before disentanglement, word expanding that treats all non-ground-truth words as negatives in alignment, and attribute smoothing that prevents the model from overshooting its confidence in seen attribute-object pairs. The ablations show that switching word embeddings from Word2Vec plus FastText to LLaVA hidden states helps not only TRIDENT but also earlier models, and that the largest single contribution to unseen accuracy comes from the soft attribute labels.","pith_inferences":["A testable extension the paper leaves implicit: ablating LLaVA v1.5's visual encoder against an ImageNet-only ViT of the same size would separate gains from richer multimodal features versus gains from pretraining data overlap, which matters for any deployment domain outside the three benchmarks.","Attribute smoothing is a general recipe: any recognition task with many-to-many attribute-object relations could generate auxiliary labels with a text LLM and soften targets, with CZSL being the setting where the benefit is easiest to measure because unseen accuracy tracks generalization directly.","The word-expanding trick, treating every non-ground-truth primitive as a negative, is compatible with other disentanglement losses and could improve compositional retrieval beyond classification.","The optimal number of visual features and smoothing factor are dataset-dependent, with more features helping on noisy MIT-States and fewer working better on cleaner C-GQA, suggesting the framework's components matter most when labels are noisy."],"forward_implications":["The best published non-CLIP CZSL result on MIT-States, C-GQA, and VAW-CZSL is now TRIDENT, with the previous CANet numbers replaced by AUC 14.2%, 8.0%, and 8.3% respectively.","Switching word embeddings from Word2Vec plus FastText to LLaVA hidden states improves not only TRIDENT but also existing models such as SCEN and CANet, so MLLM embeddings are a drop-in upgrade for word-alignment-based CZSL.","Attribute smoothing changes the training target for attributes from a single one-hot label to a soft distribution over plausible attributes, which reduces overconfidence on seen compositions and is the main driver of the unseen-accuracy gains.","The disentanglement module contributes through both local background suppression with feature adaptive aggregation and dimensional splitting of global features with condition masks, with orthogonal regularization keeping the two streams complementary.","Because the visual encoder stays frozen, TRIDENT can adopt newer MLLM visual backbones without retraining the disentanglement head, so the framework is forward-compatible with better encoders."],"supporting_citations":[{"why":"Supplies LLaVA v1.5, the frozen visual encoder and the MLLM whose last hidden states are used as word embeddings.","marker":"[Liu et al., 2024a]"},{"why":"CANet, the previous state-of-the-art baseline whose AUC and harmonic-mean figures TRIDENT is compared against and exceeds.","marker":"[Wang et al., 2023]"},{"why":"GritLM, which establishes the practice of using last hidden states of generative LLMs as representational embeddings and is the basis for TRIDENT's MLLM embeddings.","marker":"[Muennighoff et al., 2024]"},{"why":"GPT-3.5, used to generate the auxiliary attribute words that feed the attribute smoothing loss.","marker":"[OpenAI, 2023]"},{"why":"OADis and the VAW-CZSL dataset, providing one of the three benchmarks and a disentangling-by-image-pairs baseline.","marker":"[Saini et al., 2022]"},{"why":"Provides the C-GQA dataset and the graph-embedding formulation used as a baseline.","marker":"[Naeem et al., 2021]"},{"why":"Provides the MIT-States dataset, the main benchmark used for ablations and hyperparameter studies.","marker":"[Isola et al., 2015]"},{"why":"CompCos, which supplies the cosine-similarity cross-entropy alignment formulation that TRIDENT adapts with word expanding and smoothing.","marker":"[Mancini et al., 2021]"}],"fun_headline_variants":["MLLM embeddings and attribute smoothing beat CANet on 3 CZSL benchmarks","LLM-smoothed attributes and MLLM embeddings set new CZSL high scores","Replace word embeddings with MLLM hidden states to boost CZSL AUC","TRIDENT: LLM attribute smoothing lifts CZSL to new highs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that LLaVA v1.5, used frozen as the visual backbone and as the source of word embeddings, has not memorized the specific attribute-object labels or near-duplicate test images in the three benchmarks; the authors address this only in Appendix C by noting that its training objective was to produce captions about main image content, not attribute-object labels.","fun_headline_variants_meta":{"raw":{"variants":["MLLM embeddings and attribute smoothing beat CANet on 3 CZSL benchmarks","LLM-smoothed attributes and MLLM embeddings set new CZSL high scores","Replace word embeddings with MLLM hidden states to boost CZSL AUC","TRIDENT: LLM attribute smoothing lifts CZSL to new highs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000733,"raw_usage":{"total_tokens":3348,"prompt_tokens":1084,"completion_tokens":2264,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":700,"completion_tokens_details":{"reasoning_tokens":2177}},"tokens_in":700,"tokens_out":2264,"duration_ms":15652,"temperature":1.0,"reasoning_tokens":2177,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T18:37:26.133924+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the identical TRIDENT pipeline with a frozen visual encoder trained only on ImageNet-1k, with no web-scale caption data, and compare the AUC on the three benchmarks; if the margin over CANet shrinks to near zero while the rest of the architecture is unchanged, the reported gains are attributable to pretraining-data overlap with the test splits rather than to disentanglement and attribute smoothing.","supporting_citations":[{"cited_title":"Learning conditional attributes for compositional zero-shot learning","cited_arxiv_id":null,"evidence_quote":"CANet, the previous state-of-the-art baseline whose AUC and harmonic-mean figures TRIDENT is compared against and exceeds."},{"cited_title":"Gpt-3.5-turbo api, 2023","cited_arxiv_id":null,"evidence_quote":"GPT-3.5, used to generate the auxiliary attribute words that feed the attribute smoothing loss."},{"cited_title":"Disentangling visual embeddings for attributes and objects","cited_arxiv_id":null,"evidence_quote":"OADis and the VAW-CZSL dataset, providing one of the three benchmarks and a disentangling-by-image-pairs baseline."},{"cited_title":"Discovering states and transformations in image collections","cited_arxiv_id":null,"evidence_quote":"Provides the MIT-States dataset, the main benchmark used for ablations and hyperparameter studies."},{"cited_title":"Open world compositional zero-shot learning","cited_arxiv_id":null,"evidence_quote":"CompCos, which supplies the cosine-similarity cross-entropy alignment formulation that TRIDENT adapts with word expanding and smoothing."}],"review_version":1}