{"id":"5c3ddaa1-19a7-4127-942c-6d81f92d526c","arxiv_id":"2501.17171","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"Splitting text prompts into attribute, object, and pair components and fusing them with inter/intra-modality cross-attention gives small accuracy gains over DFSP on three compositional zero-shot learning benchmarks.","lead":"The paper describes a way to teach an AI to recognize never-before-seen combinations of words and images, like \"blue cat\", by giving the model separate prompts for the color, the animal, and the pair, then merging the information. It reports slightly better scores than older methods on three standard tests, though the evidence is partly inconsistent and hard to reproduce.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Tables 1-2 contain internally impossible baseline rows (Co-CGE identical across open/closed settings; CGE repeats the MIT-States row for C-GQA), so the claimed margins over baselines are not currently verifiable.","rationale":"The reader's weakest assumption is that the baseline numbers are correctly transcribed and that all methods are evaluated under an identical protocol. My independent reading confirms this is the most load-bearing premise, and the manuscript itself provides direct evidence against it: Table 2's Co-CGE row is byte-for-byte identical to Table 1's Co-CGE row across twelve values, and Table 2's CGE row repeats the MIT-States row for C-GQA. These are objective internal inconsistencies, not a matter of disagreeing with an external consensus. Since the paper's entire quantitative contribution is a small margin over DFSP and other baselines, corrupted baseline rows mean the central claim is unsupported by the presented evidence. I also checked the prose claims in Section 4.2; the claimed 36.0% AUC improvement on UT-Zappos does not match any plausible comparison of the table numbers, which strengthens the concern that the results were not carefully verified. The architectural idea is coherent and the ablation tables are internally monotone in the expected direction, so this is not a critique of the method's concept; it is a critique of the evidence quality needed to establish the empirical claim. The absence of code and the use of the test benchmark to select among 27 prompt configurations (Table 3) without a held-out validation set further prevent independent verification. For these reasons, the appropriate verdict is REJECT, matching the reader's moderate-confidence rejection. A corrected and independently verified results table, plus code or a clear validation protocol, could change the verdict to conditional acceptance or higher.","tokens_in":10272,"tokens_out":3136,"duration_ms":30996,"concrete_test":"Reconstruct Tables 1 and 2 by fetching every baseline's published results from its original paper under the same closed/open protocol, rather than trusting the manuscript's transcriptions. Pay special attention to Co-CGE, CGE, SCEN, CSP, and DFSP in closed-world settings. If the reconstructed Co-CGE closed-world row differs from the open-world row, or the reconstructed CGE C-GQA column differs from the MIT-States column, the duplicate rows are confirmed errors; then recompute Ours' margin against the corrected DFSP row for each metric. If the margin shrinks, changes sign, or falls below run-to-run variation, the central claim of consistent improvement does not survive.","verdict_should_be":"REJECT","load_bearing_attack":"The paper's central claim is that separating hard pair, soft attribute, and soft object prompts, then fusing them through inter- and intra-modality cross-attention (MFSB), improves CZSL accuracy over DFSP and other baselines. The only direct evidence for this claim is the margin over baselines in Tables 1 and 2, and those tables are internally corrupted. In Table 2, the Co-CGE row is numerically identical to the Co-CGE row in Table 1 for all three datasets and all four metrics, which cannot happen because open-world and closed-world protocols use different seen/unseen splits. Also in Table 2, the CGE row lists exactly the same four values for MIT-States and C-GQA (28.7, 25.3, 17.2, 5.1), again an impossible copy-paste artifact. Because these rows are transcribed baseline values, the corruption is not a harmless formatting issue: if one row is wrong, the entire table's reliability is in question, and the reported gains of Ours over DFSP cannot be trusted. Section 4.2 further claims AUC improvements of 5.4%, 36.0%, and 10.5% on the three datasets, but no consistent comparison of the table values yields these numbers; for example, UT-Zappos AUC in Table 1 is 29.27 vs. 26.4 for DFSP, a 10.9% relative gain, not 36.0%. No code or reproducibility artifacts are provided, and the final prompt configuration was selected from 27 ablation runs in Table 3 using the same MIT-States open-world benchmark without a described held-out validation set, so test-set information may have influenced the reported configuration. The method itself is plausible and the internal ablations are consistent, but the empirical core of the paper is not currently verifiable.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a compositional zero-shot learning (CZSL) method that separates prompts into a hard pair prompt and soft attribute/object prompts, then fuses the resulting vision-language features through a Modal Fusion Synthesizer Block (MFSB) with inter-modality followed by intra-modality cross-attention. The central claim is that this decomposition and fusion order improves CZSL accuracy over existing baselines including DFSP, on MIT-States, UT-Zappos, and C-GQA in both open-world and closed-world settings. The paper reports state-of-the-art or near-state-of-the-art results in Tables 1 and 2, supported by three ablations (prompt type, prompt components, fusion order).","tokens_in":10581,"tokens_out":2986,"duration_ms":29874,"significance":"If the reported results were reliable, the contribution would be a modest but useful incremental improvement over DFSP, with a fairly thorough ablation study of prompt types and fusion order. The idea of separating hard pair prompts from soft attribute and object prompts, and the specific inter-then-intra fusion order, is a reasonable design choice that could interest the CZSL and prompt-learning communities. However, the empirical evidence is the only support for the central claim, and that evidence is internally inconsistent: two baseline rows in Table 2 are either identical across open/closed settings or repeated across datasets, and the percentage gains quoted in Section 4.2 do not match the table values. Because the paper's contribution is empirical and the baseline table appears corrupted, the significance cannot be assessed until the numbers are corrected and verified.","major_comments":[{"comment":"The Co-CGE row in Table 2 is numerically identical to the Co-CGE row in Table 1 for all three datasets and all four metrics (e.g., MIT-States: 31.1, 5.8, 6.4, 1.1). This cannot be correct because open-world and closed-world evaluations use different seen/unseen splits, so the same baseline cannot yield identical Seen, Unseen, HM, and AUC values in both settings. Likewise, the CGE row in Table 2 lists the same four values (28.7, 25.3, 17.2, 5.1) for MIT-States and C-GQA. These are copy-paste artifacts that call into question the integrity of the entire baseline comparison. Since the central claim of superiority over DFSP and other methods rests entirely on the margins in these tables, the paper's main result is currently unverifiable.","section":"Tables 1 and 2"},{"comment":"The text claims: \"Ours outperforms in AUC scores of 5.4% on MIT-States, 36.0% on UT-Zappos, and 10.5% on CGQA, surpassing by 4.3%\" and \"6.8% increase in the harmonic mean on the MIT-States.\" No consistent interpretation of Table 1 yields these numbers. For example, UT-Zappos AUC in Table 1 is 29.27 (Ours) vs. 26.4 (DFSP), which is a relative improvement of about 10.9%, not 36.0%. MIT-States HM is 20.35 vs. 19.1, a relative gain of about 6.5%, not 6.8%. The claimed percentages are unsupported by the tables; the reader cannot reconstruct the reported gains, which is a serious problem for a results section.","section":"Section 4.2, Results"},{"comment":"The final prompt configuration (Hard Pair, Soft Object, Soft Attribute) and the loss weights alpha=beta=gamma=0.2 appear to be selected from the 27 runs in Table 3, which are all evaluated on the MIT-States open-world benchmark. The paper does not describe a held-out validation set or any procedure for choosing hyperparameters without using the test split. Consequently, the reported final numbers may partially reflect test-set information, and the claimed improvements over baselines that do not receive such tuning are not fairly compared. This is a load-bearing methodological issue for the paper's empirical claim.","section":"Section 3.5 and Table 3"},{"comment":"No code, random seeds, or implementation details of the cross-attention layers (e.g., number of heads, dimension of Q/K/V, attention dropout) are provided. The loss combination in Section 3.5 uses unclear notation (e.g., L_pair_soft, L_attr_soft, L_obj_soft are referenced but not all defined). Without these details, the ablations in Tables 3-5 cannot be reproduced independently, further reducing the confidence in the reported margins.","section":"General reproducibility"}],"minor_comments":[{"comment":"The dataset description mixes up UT-Zappos and MIT-States: after describing MIT-States, the text says \"The dataset has 50,025 shoe images labeled under 12 categories and 16 attributes,\" which is the UT-Zappos description, but appears in the MIT-States paragraph. This should be corrected.","section":"Section 4.1"},{"comment":"Several references are garbled or duplicated. For example, references [10], [11], [12], and [13] all appear to describe the same 'Learning graph embeddings for compositional zero-shot learning' paper with different venue strings. Reference [1] is misattributed to Apley and Zhu instead of the intended CZSL work. The reference list needs a thorough revision.","section":"References"},{"comment":"The notation in Sections 3.4.1 and 3.4.2 is inconsistent: 'inter-fused' and 'intra-fused' features share superscripts that are easy to confuse (e.g., v_pair_intra1 vs. v_attr_intra2). The definitions of losses like L_pair_soft are not given explicitly; they are introduced only in the total loss equation. Please define all terms before use.","section":"Notation"},{"comment":"Figure 1 and Figure 2 are referenced but not included in the text; the captions suggest they are important for understanding the fusion layout. If the figures are present in the actual submission, ensure they are visible and legible; otherwise, the text alone is insufficient to follow the architecture.","section":"Figures"}],"recommendation":"reject","confidential_remarks":"The reader's report and the stress-test concern are confirmed by direct inspection: the identical Co-CGE rows across Tables 1 and 2, and the repeated CGE row for MIT-States and C-GQA in Table 2, are damning internal inconsistencies. This is not a minor formatting issue; it undermines the entire baseline comparison on which the paper's contribution rests. The percentage claims in Section 4.2 are also not reproducible from the tables. Unless the authors can provide corrected tables and a reproducible evaluation protocol, the central claim cannot be verified. The paper also appears to be an early draft, with garbled references and unclear notation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take, before anything else: the central comparison is not verifiable as printed. I checked the tables and the stress-test is right. Table 2's Co-CGE row is identical to Table 1's, which cannot happen across open/closed splits, and the CGE row for C-GQA copies MIT-States values. Once one row is a copy-paste artifact, every margin against baselines in this paper is suspect.\n\nOn the plus side, the architectural idea is coherent. Splitting the prompt into a hard pair prompt plus soft attribute and object prompts, then fusing through inter-modality then intra-modality cross-attention, is a reasonable extension of DFSP. The paper also does a thorough ablation: 27 prompt configurations in Table 3, component ablations in Table 4, fusion-order ablations in Table 5. Those internal comparisons are internally consistent and tell a coherent story about which design choices help. If the final numbers could be trusted, this would be a modest, publishable increment.\n\nBut the load-bearing numbers do not hold up. Beyond the table corruption, the text claims AUC improvements of 5.4%, 36.0%, and 10.5% on MIT-States, UT-Zappos, and C-GQA. From Table 1, the relative AUC gain over DFSP on UT-Zappos is about 10.9%, not 36%. So the prose is wrong. There is no code or reproducibility artifact, and the final prompt configuration was selected from the 27 ablation runs using the same MIT-States open-world benchmark, with no described held-out split. That is a mild selection-on-test issue, but combined with the table problems it means the empirical core is unverifiable.\n\nWhere does this leave us? The idea is plausible and the ablations are informative, but the central claim collapses until the baseline table is fixed and the numbers reconciled. I would not spend referee time on this version. If the author releases code and corrects the tables, it might become a minor but solid CZSL paper. As is, it should be sent back or rejected. My recommendation: desk reject under current evidence, or at most a 'revise and resubmit' with a hard requirement for corrected tables, code, and a description of how the prompt configuration was selected.","headline":"The architecture is plausible but the empirical core is unverifiable: baseline tables are corrupted and prose gains don't match the numbers.","tokens_in":11141,"tokens_out":2214,"would_cite":false,"duration_ms":22735,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that separating the text prompt into hard pair, soft attribute, and soft object prompts, then fusing text and image features first across modalities and then within modalities, improves compositional zero-shot recognition…","keywords":["compositional zero-shot learning","prompt learning","vision-language models","cross-attention","attribute recognition","multi-modal fusion","MIT-States","UT-Zappos"],"falsifier":"A re-implementation that runs the paper's exact training recipe (CLIP ViT-L/14, 20 epochs, Adam, same splits and loss weights) and the DFSP baseline under identical conditions would settle the claim: if DFSP reaches or exceeds the paper's reported HM and AUC values on MIT-States and UT-Zappos, the separated-prompt margin is not reproducible. A cheaper check is to verify whether the Co-CGE row in the closed-world table, which duplicates the open-world Co-CGE row, matches the originally published Co-CGE numbers.","tokens_in":10034,"feed_emoji":"🧩","tokens_out":8602,"duration_ms":81029,"temperature":0.7,"pith_summary":"Compositional zero-shot learning asks a model to recognize unseen pairings of an attribute and an object, such as a blue cat, after training on other combinations. The paper's central claim is that the standard recipe of one joint text prompt—typically a hard prompt like 'a photo of [state] [object]'—leaves the model blind to the roles of the two parts, and that separating the prompt into a hard pair prompt plus soft attribute and soft object prompts, then fusing the resulting text and image features across modalities and then within each modality, improves recognition. The authors report that this design, called MFSB with separated prompts, beats the DFSP baseline and earlier methods on MIT-States, UT-Zappos, and C-GQA under both open-world and closed-world settings, with the largest reported gains on unseen compositions and AUC. If these results hold, the contribution is a concrete prompt-engineering and feature-fusion recipe that can be dropped into any CLIP-based CZSL system.","feed_headline":"Three-way prompt split with cross-modal fusion beats CZSL baselines","feed_subtitle":"Three prompt types, fused across and within text and image streams, outperform prior zero-shot composition methods.","key_machinery":"The load-bearing structure is the set of three prompt-conditioned text features and their corresponding image features. A hard pair prompt $P_{hard}^{pair}=\\{a\\,photo\\,of,x_s,x_o\\}$ feeds both state and object labels at once, while a soft attribute prompt $P_{soft}^{attr}=\\{x_1,\\dots,x_k,x_s\\}$ and a soft object prompt $P_{soft}^{obj}=\\{x_1,\\dots,x_k,x_o\\}$ each prepend learnable tokens to a single label. The Modal Fusion Synthesizer Block (MFSB) applies cross-attention twice: inter-modality fusion alternates text and image features as query, key, and value to merge the two modalities, and intra-modality fusion then uses the attribute feature and object feature of the same modality as key and value for each other. The losses from the pair, attribute, and object branches at the initial, inter-fused, and intra-fused stages are combined into a total loss with weights $\\alpha=\\beta=\\gamma=0.2$. This machinery is what lets the attribute and object each keep their own text representation while forcing the image and text sides to refine each other twice, which the paper argues captures subtle semantic differences.","core_discovery":"The core discovery the paper tries to establish is that prompt decomposition changes what a vision-language model can extract from an image-text pair. Instead of feeding a single pair prompt 'a photo of [state] [object]' into the CLIP text encoder, the method keeps that hard pair prompt but adds two learnable soft prompts, one for the state and one for the object. The resulting text features and the image features are passed through a Modal Fusion Synthesizer Block: first a cross-attention layer mixes text and image features for each prompt (inter-modality fusion), then a second cross-attention layer lets the state feature and object feature exchange information of the same type (intra-modality fusion). The paper argues that this two-stage, separated-prompt design outperforms the single-prompt baseline and several prior CZSL methods, and that the best configuration is hard pair, soft object, soft attribute with inter-fusion before intra-fusion.","pith_inferences":["A natural testable extension is to apply the same separated-prompt and two-stage fusion recipe to other CLIP-based composition tasks, such as human-object interaction detection, where the paper's ablation suggests the pattern would transfer; the paper does not run those experiments.","Because the prompt-form search in Table 3 is only run on MIT-States, the claim that hard pair plus soft object plus soft attribute is universally the best configuration across datasets is an extrapolation the paper has not directly verified.","If the loss weights $\\alpha=\\beta=\\gamma$ were made learnable instead of fixed at 0.2, the method might adaptively balance the initial, inter-fused, and intra-fused losses in open-world versus closed-world settings; this is a modification the paper does not explore."],"forward_implications":["On the three benchmarks used, the method reports the best or near-best seen, unseen, harmonic mean, and AUC scores in both open-world and closed-world settings, making it the stated new reference point among the compared CZSL methods.","The ablation tables show that all three prompt components contribute: pair-only is the weakest configuration, and adding separated object and attribute prompts raises harmonic mean and AUC on MIT-States.","The fusion-order ablation identifies inter-modality fusion followed by intra-modality fusion as the best arrangement, giving future systems a concrete ordering rule for cross-attention fusion blocks.","Because the added components are only prompts and cross-attention layers wrapped around a frozen CLIP encoder, the method can be grafted onto any CLIP-based CZSL pipeline without retraining the backbone."],"supporting_citations":[{"why":"This is the baseline the paper directly extends; it supplies the hard pair prompt and the decomposing/fusing structure that the separated prompts and MFSB replace.","marker":"[8]"},{"why":"Defines the open-world CZSL evaluation and serves as a strong non-prompt baseline in the comparison tables.","marker":"[9]"},{"why":"Provides the Co-CGE baseline numbers that appear in both open-world and closed-world tables.","marker":"[10]"},{"why":"Provides the CGE baseline used as a comparison point.","marker":"[11]"},{"why":"Introduces the soft-prompt composition baseline CSP, which the paper compares against and extends.","marker":"[17]"},{"why":"Supplies the MIT-States dataset and its attribute/object split, the main benchmark for all ablations.","marker":"[4]"},{"why":"Supplies the C-GQA dataset used as the third evaluation benchmark.","marker":"[13]"},{"why":"Supplies the UT-Zappos dataset used as the second evaluation benchmark.","marker":"[28]"},{"why":"Establishes soft prompt tuning for vision-language models, the source of the soft separated prompt idea.","marker":"[30]"}],"fun_headline_variants":["Separate prompts, fused features boost zero-shot composition","Hard pair, soft attribute/object: fused prompts win CZSL","Inter&intra modal fusion with separated prompts beats CZSL","Prompt split and dual fusion lift compositional zero-shot learning"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire claimed advantage is measured against baseline numbers in Tables 1 and 2, so the central claim collapses if those numbers were transcribed incorrectly or if the comparison protocol differed between methods.","fun_headline_variants_meta":{"raw":{"variants":["Separate prompts, fused features boost zero-shot composition","Hard pair, soft attribute/object: fused prompts win CZSL","Inter&intra modal fusion with separated prompts beats CZSL","Prompt split and dual fusion lift compositional zero-shot learning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000619,"raw_usage":{"total_tokens":2820,"prompt_tokens":839,"completion_tokens":1981,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":455,"completion_tokens_details":{"reasoning_tokens":1911}},"tokens_in":455,"tokens_out":1981,"duration_ms":14413,"temperature":1.0,"reasoning_tokens":1911,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T16:59:45.681972+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A re-implementation that runs the paper's exact training recipe (CLIP ViT-L/14, 20 epochs, Adam, same splits and loss weights) and the DFSP baseline under identical conditions would settle the claim: if DFSP reaches or exceeds the paper's reported HM and AUC values on MIT-States and UT-Zappos, the separated-prompt margin is not reproducible. A cheaper check is to verify whether the Co-CGE row in the closed-world table, which duplicates the open-world Co-CGE row, matches the originally published Co-CGE numbers.","supporting_citations":[],"review_version":1}