{"id":"48f5cf2f-96dc-499d-8a1b-246517f266c5","arxiv_id":"2506.20986","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":8,"one_line_summary":"EVA reports SOTA on MIT-States, UT-Zappos, and C-GQA in closed- and open-world CZSL by combining MoE adapters with semantic variant alignment, but missing backbone-matched baselines weaken the claim.","lead":"EVA adds mixture-of-experts adapters and a semantic variant alignment module to frozen CLIP to improve compositional zero-shot learning, the recognition of never-seen state-object pairs. The paper reports state-of-the-art scores on three benchmarks, but the comparisons use a larger CLIP backbone than the baselines, so the claimed gains are not yet established.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Backbone mismatch with published baselines undermines the SOTA claim: EVA uses CLIP ViT-L/14 while the compared baselines were evaluated with ViT-B/16, and no same-backbone rerun is provided.","rationale":"The reader's weakest_assumption identifies the same load-bearing issue. The paper's own ablation is the strongest evidence in its favor: Table 4 shows Domain-expert Adaption plus Semantic Variant Alignment versus a BASELINE under the same ViT-L/14 backbone, with AUC gains on MIT-States (20.2 to 24.0) and C-GQA (10.4 to 18.8). However, the abstract and Section 4.2 make the stronger claim of outperforming SoTA baselines. Those baselines were published with ViT-B/16. The paper does not state baseline backbones in Tables 1 and 3, nor does it rerun them with ViT-L/14. Since a stronger backbone alone can produce large CZSL gains, the cross-method margin is unproven. The technical proposal is plausible, but the central empirical claim of state-of-the-art performance is not adequately supported. A matched-backbone re-run would settle the question, and the reader's REJECT verdict is consistent with this concern.","tokens_in":14221,"tokens_out":4561,"duration_ms":50597,"concrete_test":"Run EVA and every compared baseline (DFSP, CDS-CZSL, Troika, RAPR, PLID, CSP) with the same frozen CLIP ViT-L/14 backbone, same prompt/capacity budget, and same train/validation/calibration protocol on MIT-States, UT-Zappos, and C-GQA; alternatively, run EVA with ViT-B/16 and compare to the published ViT-B/16 tables. If the closed-world AUC margins shrink to roughly noise level or invert, the SOTA claim is not established. If the margins persist under a matched backbone, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"EVA's central assertion is state-of-the-art CZSL performance (Tables 1 and 3). The load-bearing assumption is that the comparison to prior methods is fair. The paper trains EVA on frozen CLIP ViT-L/14 (Section 4.1, Implementation Details), but the cited SoTA results (DFSP [25], CDS-CZSL [21], Troika [9], RAPR [13], PLID [4], etc.) were obtained with the smaller CLIP ViT-B/16 backbone. EVA never reruns any baseline with ViT-L/14, nor does it report EVA with ViT-B/16. Because ViT-L/14 has substantially higher capacity for zero-shot visual recognition, the reported closed-world AUC gains (+1.5%, +5.7%, +4.4%) and open-world gains (+1.2%, +2.9%, +1.2%) may be mostly backbone-driven rather than attributable to domain-expert adaption and semantic variant alignment. Table 4's internal ablation does show the components improve over a same-backbone BASELINE, so the method has some internal support; however, that does not establish superiority over other methods under a matched backbone. The absence of code and error bars makes the cross-paper comparison unverifiable as reported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes EVA, a compositional zero-shot learning method built on frozen CLIP encoders. It inserts LoRA-based Mixture-of-Experts adapters into the image and text encoders (domain-expert adaption) and adds two alignment regularizers: text-to-image alignment aggregates composition probabilities into state and object scores, and image-to-text alignment selects among the encoder's expert feature variants using inter- and intra-model affinity. The training objective combines composition, state, and object cross-entropy losses, and inference adds the state and object scores to the composition logits. The paper reports state-of-the-art results on MIT-States, UT-Zappos, and C-GQA in both closed- and open-world settings, with ablations on the two main components.","tokens_in":14491,"tokens_out":6351,"duration_ms":68172,"significance":"The proposed architecture is conceptually clean, and the internal ablation (Table 4) is consistent: both domain-expert adaption and semantic variant alignment improve over a same-backbone prompt-only baseline, and the gains co-occur. If the claims were established under a matched backbone, the token-level MoE adapter and variant-selection alignment would be a useful contribution to CZSL. However, the headline comparison to previous methods is not currently valid because the cited baselines were evaluated with a weaker visual backbone than EVA. The paper's claim to state-of-the-art performance is therefore not established by the present evidence; the method's internal gains are real but smaller than the reported SOTA margins. The paper also does not report error bars or release code, which limits verification of the empirical claims.","major_comments":[{"comment":"EVA is built on frozen CLIP ViT-L/14 (Section 4.1, Implementation Details), while the published numbers quoted for DFSP, CDS-CZSL, Troika, PLID, RAPR, and the other baselines were obtained with the CLIP ViT-B/16 backbone. The paper neither reruns any baseline with ViT-L/14 nor reports EVA with ViT-B/16. This makes the claimed closed-world AUC gains (+1.5%, +5.7%, +4.4%) and open-world AUC gains (+1.2%, +2.9%, +1.2%) not attributable to the proposed method. Table 4 supports this concern: the prompt-only BASELINE with ViT-L/14 already reaches an AUC of 20.2 on MIT-States, which is essentially at the level of published ViT-B/16 DFSP (20.6–20.8) and close to CDS-CZSL (22.4). Please provide same-backbone comparisons, either by rerunning the baselines under ViT-L/14 or by training EVA under ViT-B/16, before the SOTA claim can be evaluated.","section":"Section 4.1; Tables 1 and 3"},{"comment":"The image-to-text alignment selects a feature variant by arg max over the affinity scores A_S and A_O, but the paper does not state how gradients are passed through this discrete selection for the losses L_v^s and L_v^o in Eqs. (17)–(18). Since the selected variant is used as f_s and f_o in the subsequent probability computations, the training dynamics depend crucially on this implementation (e.g., straight-through estimation, a softmax relaxation, or detached routing). Please specify and justify the differentiable implementation. If a hard arg max is used with no gradient to non-selected experts, the router and expert outputs in the image encoder receive no learning signal from this loss, which should be discussed explicitly.","section":"Section 3.4, Eqs. (13)–(14)"},{"comment":"The paper reports single-run numbers with no standard deviations or number of seeds. Many of the decisive margins are small in the harder settings, such as open-world C-GQA (AUC 5.6 vs. 4.4 for RAPR) and MIT-States closed-world (AUC 24.0 vs. 22.5 for CDS-CZSL). Given the backbone mismatch and the absence of variance estimates, the claim of 'significantly outperforms other state-of-the-art CZSL methods' is not supported. Please report mean and standard deviation over at least three seeds for EVA and, if feasible, for the baselines rerun under the same backbone.","section":"Tables 1 and 3; Section 4.2"}],"minor_comments":[{"comment":"The notation is imprecise: f_s and f_o are written as the result of an arg max over affinity scores, but in Eqs. (15)–(16) they are used as feature vectors. Write f_s = v_{i^*_s} with i^*_s = arg max_i a^s_i, and similarly for f_o.","section":"Eqs. (13)–(14)"},{"comment":"The text refers to 'RAPR [21]' when comparing on UT-Zappos; RAPR is reference [13], while [21] is CDS-CZSL. Please correct the citation.","section":"Section 4.2, Open-World paragraph"},{"comment":"The prose states that C-GQA has 453 states and 870 objects, while Table 2 lists 413 states and 674 objects. Please reconcile this discrepancy.","section":"Section 4.1 vs. Table 2"},{"comment":"The phrase 'to access the effect of key components' should be 'to assess the effect'; this appears to be a typo.","section":"Section 4.3"},{"comment":"The trainable coefficients tau_s and tau_o in Eqs. (7)–(8) are not given initial values or constraints in Section 4.1. Please specify them, since they directly scale the state and object probabilities used in the loss.","section":"Section 4.1; Eqs. (7)–(8)"},{"comment":"The numbers in Table 2 run together in the rendered text (e.g., '281751262' and '278362'); add clear column separators so the dataset split statistics are readable.","section":"Table 2"}],"recommendation":"major_revision","confidential_remarks":"I do not join the reader's recommendation of outright rejection. The backbone mismatch is serious and invalidates the SOTA claim as written, but the fix is feasible within the paper's scope: rerun baselines under ViT-L/14, or train EVA under ViT-B/16, and add error bars. The arg max differentiation question is also addressable. The internal ablations suggest a genuine effect from the proposed components, so major revision rather than rejection is appropriate."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Bottom line: there is a real idea here and the paper is honestly written, but the central empirical claim is not supported as reported. EVA uses frozen CLIP ViT-L/14 while the cited baselines (DFSP, Troika, CDS-CZSL, RAPR) were evaluated with ViT-B/16, and the paper never reruns them under a matched backbone. The stress-test note is correct, and it is load-bearing: the '+1.5% to +5.7%' gains may be mostly backbone-driven. The prompt-only BASELINE in Table 4 already hits 20.2 AUC on MIT-States with ViT-L/14, close to several published ViT-B/16 numbers, which shows how much of the margin can come from the encoder alone.\n\nWhat is actually new: the combination of LoRA-based MoE adapters with a shared expert and routed experts, inserted into frozen CLIP layers, plus a global-to-local alignment that selects among expert feature variants for state/object matching. That combination is new in CZSL. The method is well-motivated, the math for the losses is clear (aside from one issue below), and the ablations are genuinely useful — they show each component contributes on a same-backbone baseline, which is real evidence for the method's internal validity.\n\nSoft spots, in proportion. The backbone mismatch is the biggest issue and breaks the 'state-of-the-art' claim as stated. The paper needs same-backbone baselines or at least a ViT-B/16 run of EVA to make the comparison meaningful. Second, the arg-max variant selection in Eqs. 13–14 is non-differentiable, and the paper never explains how gradients reach the router or the variant representations during training. That is not a minor detail; it affects whether the method trains as described. Third, there are no error bars, no code, and no seeds reported, which makes the cross-paper numbers hard to trust even beyond the backbone issue. I would not call these fatal flaws — the internal ablation carries much of the weight — but they are concrete gaps.\n\nThe citation pattern is fine: the paper engages the relevant CZSL literature and does not overclaim in the related work section. The circularity concern is a non-issue; the text-to-image alignment is a consistency regularizer, not a circular fit to test labels. The token-load analysis is suggestive but thin, so I would not rely on it for the specialization claim.\n\nWho gets value: CZSL researchers working on adapter-based CLIP tuning or cross-modal alignment. It deserves a serious referee, because the idea is sound and the ablations suggest it works — but the referee should require major revisions: matched-backbone comparisons, code release, error bars, and a clear treatment of the arg-max differentiation. My verdict on the current version is skeptical, not outright reject.","headline":"A plausible and cleanly written method with solid internal ablations, but the central SOTA claim is undermined by comparing against baselines run on a smaller backbone.","tokens_in":15058,"tokens_out":2464,"would_cite":false,"duration_ms":28967,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"EVA claims top results in compositional zero-shot learning with MoE adapters and variant alignment","keywords":["compositional zero-shot learning","mixture-of-experts","semantic variant alignment","vision-language models","CLIP","LoRA adapter","open-world recognition","fine-grained attribute-object recognition"],"falsifier":"Rerun the cited baselines (DFSP, Troika, CDS-CZSL, RAPR, and others) with the same frozen CLIP ViT-L/14 backbone and the same 20-epoch training protocol used for EVA; if their AUC and harmonic-mean scores rise to match EVA's, the central claim of superiority is falsified.","tokens_in":13980,"feed_emoji":"🧩","tokens_out":5585,"duration_ms":53448,"temperature":0.7,"pith_summary":"The paper tries to establish that compositional zero-shot learning—recognizing unseen state-object pairs such as \"blue grass\" from learned state and object concepts—is improved by replacing the uniform composition-to-prototype mapping used in prior work with two mechanisms: token-aware mixture-of-experts adapters and semantic variant alignment. On MIT-States, UT-Zappos, and C-GQA, the proposed EVA framework reports AUC gains of +1.5, +5.7, and +4.4 points in the closed-world setting, and +1.2, +2.9, and +1.2 points in the open-world setting. If these results hold, they would show that expert specialization and instance-dependent primitive matching, built on frozen CLIP encoders, are effective ingredients for compositional generalization.","feed_headline":"EVA beats prior CZSL methods by up to 5.7 AUC points","feed_subtitle":"Mixture-of-experts adapters plus variant-aware matching lift accuracy on MIT-States, UT-Zappos, and C-GQA.","key_machinery":"The central mechanism is the MoE adapter combined with the variant-selection alignment. The adapter, placed at every encoder layer, uses a router to assign each token to one shared expert and K routed experts; the experts are LoRA low-rank projections, keeping the parameter cost small while letting different experts specialize in state-related or object-related semantic content. The semantic variant alignment then operates globally to locally: from the text view, primitive probabilities are computed by taking the maximum over composition probabilities sharing that primitive; from the image view, the CLS tokens from all experts in the final image layer are treated as alternative visual descriptions, and the variant with the highest inter-model (variant-to-primitive) plus intra-model (variant-to-composition) affinity score is chosen as the primitive visual feature. This replaces the all-to-one image-primitive alignment that the paper argues disrupts the topological structure of cross-modal fine-grained associations.","core_discovery":"The central claim is that primitive representations in CZSL should be modeled as clusters of semantic variants, not single centroids, and that the tokens carrying those variants should be routed to specialized experts. EVA inserts LoRA-based MoE adapters in parallel with the feed-forward network of every layer of the frozen CLIP image and text encoders, with a shared expert capturing common knowledge and K=2 routed experts chosen per token by a router. For alignment, text-to-image matching scores a state or object by the maximum composition score over all compositions sharing that primitive, while image-to-text matching treats the final-layer CLS tokens from the different experts as feature variants and selects the variant with highest combined affinity to the corresponding primitive text feature and to the composition feature. The paper reports state-of-the-art results on all three benchmarks in both closed- and open-world settings, with the largest margins on UT-Zappos (AUC 50.2 vs. 44.5 for the best prior method) and C-GQA (AUC 18.8 vs. 14.4).","pith_inferences":["Because the paper uses the larger CLIP ViT-L/14 backbone while the cited baselines report results with ViT-B/16, a matched-backbone rerun of the baselines is needed to separate method gains from backbone gains; the stated margins may shrink once baselines are rerun under the same backbone.","The implicit assumption that compositions sharing a primitive form a cluster centered on the primitive text feature is testable on other attribute-object datasets and on retrieval tasks, where the selected variant could be evaluated directly against ground-truth attributes.","The expert-load analysis suggests the router learns a division of labor between state and object domains; extending the same adapter design to other vision-language problems that require attribute-object disentanglement may be a direct payoff.","Scaling the number of experts or using patch-level rather than CLS-token variants could push the variant-selection mechanism further, since its capacity is bounded by the number and diversity of expert outputs."],"forward_implications":["If the reported numbers are correct, token-level MoE adapters are a parameter-efficient way to adapt frozen CLIP encoders for CZSL without adding suffix modules or unfreezing the backbone.","The variant-based alignment implies that primitive matching should depend on the instance, not on a single fixed prototype, which is a different inductive bias from most existing CZSL alignments.","The larger gains on UT-Zappos and C-GQA suggest the benefit grows with fine-grained label spaces and larger composition sets.","The open-world AUC gains of +1.2 to +2.9 indicate that the alignment transfers to the full Cartesian-product space, not just the curated test split.","The ablations attribute most of the improvement to domain-expert adaption, with semantic variant alignment adding a further gain on top, supporting the claim that both components are needed."],"supporting_citations":[{"why":"CLIP is the frozen image and text encoder backbone that EVA builds on.","marker":"[35]"},{"why":"DFSP provides the decomposed soft-prompt fusion baseline and the strongest prior comparison on several benchmarks.","marker":"[25]"},{"why":"Troika is the cross-modal traction baseline whose all-to-one alignment EVA directly contrasts.","marker":"[9]"},{"why":"RAPR is the retrieval-augmented primitive representation baseline that EVA surpasses on UT-Zappos.","marker":"[13]"},{"why":"CDS-CZSL is a context-based diversity baseline used in the closed- and open-world comparisons.","marker":"[21]"},{"why":"LoRA provides the low-rank expert structure used inside the MoE adapters.","marker":"[8]"},{"why":"The sparsely-gated mixture-of-experts layer supplies the routing mechanism that EVA adapts.","marker":"[36]"},{"why":"C-GQA is the largest evaluation dataset and also the source of the graph-embedding CZSL baseline.","marker":"[30]"},{"why":"This work defines the open-world compositional zero-shot setting and the full-composition evaluation space used for the open-world experiments.","marker":"[26]"}],"fun_headline_variants":["EVA: MoE adapters and variant alignment boost CZSL","EVA models primitives as semantic variants, sets new CZSL records","EVA's expert-routed variant alignment beats CZSL baselines","For compositional zero-shot, EVA's variant MoE outperforms"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The comparison assumes that the published baseline numbers, produced with the smaller CLIP ViT-B/16 encoder, are directly comparable to EVA's results using the larger frozen CLIP ViT-L/14, and that the reported margin reflects the proposed components rather than the backbone.","fun_headline_variants_meta":{"raw":{"variants":["EVA: MoE adapters and variant alignment boost CZSL","EVA models primitives as semantic variants, sets new CZSL records","EVA's expert-routed variant alignment beats CZSL baselines","For compositional zero-shot, EVA's variant MoE outperforms"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000192,"raw_usage":{"total_tokens":1347,"prompt_tokens":949,"completion_tokens":398,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":565,"completion_tokens_details":{"reasoning_tokens":319}},"tokens_in":565,"tokens_out":398,"duration_ms":4449,"temperature":1.0,"reasoning_tokens":319,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T22:36:20.828991+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Rerun the cited baselines (DFSP, Troika, CDS-CZSL, RAPR, and others) with the same frozen CLIP ViT-L/14 backbone and the same 20-epoch training protocol used for EVA; if their AUC and harmonic-mean scores rise to match EVA's, the central claim of superiority is falsified.","supporting_citations":[{"cited_title":"Learn- ing transferable visual models from natural language super- vision","cited_arxiv_id":null,"evidence_quote":"CLIP is the frozen image and text encoder backbone that EVA builds on."},{"cited_title":"De- composed soft prompt guided fusion enhancing for compo- sitional zero-shot learning","cited_arxiv_id":null,"evidence_quote":"DFSP provides the decomposed soft-prompt fusion baseline and the strongest prior comparison on several benchmarks."},{"cited_title":"Troika: Multi-path cross-modal trac- tion for compositional zero-shot learning","cited_arxiv_id":null,"evidence_quote":"Troika is the cross-modal traction baseline whose all-to-one alignment EVA directly contrasts."},{"cited_title":"Retrieval-augmented primitive representations for composi- tional zero-shot learning","cited_arxiv_id":null,"evidence_quote":"RAPR is the retrieval-augmented primitive representation baseline that EVA surpasses on UT-Zappos."},{"cited_title":"Context-based and diversity-driven specificity in compositional zero-shot learning","cited_arxiv_id":null,"evidence_quote":"CDS-CZSL is a context-based diversity baseline used in the closed- and open-world comparisons."},{"cited_title":"Learning graph embeddings for compositional zero-shot learning","cited_arxiv_id":null,"evidence_quote":"C-GQA is the largest evaluation dataset and also the source of the graph-embedding CZSL baseline."},{"cited_title":"Open world compositional zero- shot learning","cited_arxiv_id":null,"evidence_quote":"This work defines the open-world compositional zero-shot setting and the full-composition evaluation space used for the open-world experiments."}],"review_version":1}