{"id":"6158aa13-03d3-440c-93ce-e678b90e8901","arxiv_id":"2411.16752","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"IP-CIR creates imagined proxy images from a query image and caption via LLM-based layout and conditional generation, then blends proxy, query, and text features to improve zero-shot composed image retrieval.","lead":"This paper introduces IP-CIR, a training-free method that generates 'proxy' images matching a query image and a text description, then uses those proxy images to improve zero-shot composed image retrieval. On standard benchmarks the method lifts retrieval accuracy, for example raising CIRR Recall@10 from 77.25 to 80.00 when added to the LDRE baseline.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper never isolates the imagined-proxy contribution from the query-feature and semantic-perturbation terms plus the balance metric; the reported gains may not be caused by the proxy at all.","rationale":"The reader's weakest assumption is that MIGC++-generated proxies preserve query attributes while applying caption edits, and the reader notes that proxy-only features can hurt. I agree that proxy quality is important, but the more load-bearing gap is that the ablation design cannot attribute the final gains to the proxy at all. In Table 3, row 2 shows PI alone is neutral or slightly negative; row 3 adds RP, but RP is defined to include fp, so it never isolates fp; row 4 adds BM. A reader cannot tell whether the +3.10 CIRR R@1 and +2.02 CIRCO mAP@10 come from the proxy image, from query features and semantic perturbation, or from the balance metric re-ranking. This is particularly acute because for CIRR λ=0.0, so the final score reduces to St*Sp, meaning any image-side similarity correlated with the target could yield a similar product re-ranking. The appendix's Fig. 1 includes Qi, Si, and Pi singly and in combinations, but a direct Qi+Si without Pi control is not reported, and Table 3 has no RP-without-PI row. The paper also misstates the headline CIRR result in the abstract: 70.07 is Recall@5, not Recall@10; the actual R@10 is 80.00. This strengthens the need for care but is secondary to the missing control. I would keep the reader's conditional verdict: the method is interesting but the central causal claim needs one clean ablation before acceptance.","tokens_in":14147,"tokens_out":7328,"duration_ms":74041,"concrete_test":"Re-run the CIRR and CIRCO experiments with Eq. 1 modified so that fp is replaced by (a) the CLIP feature of the original query image and (b) an image generated from the LLM layout by a text-to-image model without the query-image conditioning/ELITE branch, keeping fq, fs, λ, and all other settings identical. If either control reproduces most of the reported gains (e.g., at least 80% of the +3.10 CIRR R@1 and +2.02 CIRCO mAP@10 improvements), then the imagined proxy is not the causal ingredient; if both controls fail to improve over the baseline, the proxy claim is supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that imagined proxy images add fine-grained visual information that improves zero-shot CIR. Table 3 shows that using proxy similarity directly (PI) slightly hurts CIRR R@1 (-0.13) and CIRCO mAP@5 (-0.06), while the full gain appears only after adding the robust proxy feature (Eq. 1: fRP = fp + max(fp)/max(fq) fq + max(fp)/max(fs) fs) and the balance metric (Eq. 2). Because Eq. 1 always contains fp, no row in Table 3 or Fig. 5 compares RP-without-fp against RP-with-fp. The improvements could therefore come from the fq + fs terms and the product-style re-ranking Sf = St*Sp (which is exact when λ=0.0 for CIRR), with the proxy image contributing little or nothing. This is not merely a proxy-quality issue; it bears directly on whether the imagined proxy is the causal ingredient, which is the paper's claimed novelty. A control that replaces fp with the query image's own CLIP feature, or with an image generated without query-image conditioning, would settle whether the expensive MIGC++ generation step is necessary.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes IP-CIR, a training-free method for zero-shot composed image retrieval (ZS-CIR). The method first uses BLIP-2 captions and an LLM to infer an object layout from the query image and relative caption, then uses MIGC++ conditioned on the query image to generate multiple 'proxy' images. A robust retrieval feature fRP is formed by adding the query image feature and a semantic perturbation (difference between LLM-generated target captions and query captions) to the proxy feature, and a balance metric combines text-based similarity and proxy-based similarity via Sf = λSt + (1−λ)StSp. The method is evaluated on CIRR, CIRCO, and FashionIQ when added to LDRE and LinCIR baselines, with reported improvements and state-of-the-art results on CIRR. The paper claims the imagined proxy provides fine-grained visual information that text-only features miss.","tokens_in":14392,"tokens_out":4555,"duration_ms":40773,"significance":"The idea of using conditional image generation to create a proxy retrieval feature is original, and the plug-and-play nature of the method is attractive. The paper uses standard datasets and baselines, and the qualitative examples show plausible cases where the proxy captures attributes that text-only retrieval misses. However, the current experiments do not isolate the proxy's causal contribution: the ablations in Table 3 show that direct proxy similarity alone does not help, and the full gain appears only with the robust feature and balance metric, both of which include query-image and semantic-perturbation terms. Without a control that removes the proxy feature, the claimed central mechanism remains unverified. The reported CIRR headline metric is also misstated in the abstract. These issues are fixable with additional experiments and corrections, so the paper's potential contribution is meaningful but not yet established.","major_comments":[{"comment":"The ablation does not isolate the contribution of the imagined proxy. The row 'PI' (direct proxy similarity) shows a small decrease for CIRR R@1 (-0.13) and CIRCO mAP@5 (-0.06), whereas the full gain appears only after adding the robust proxy feature fRP (which always includes fp) and the balance metric Sb = St*Sp. Because every ablation with RP includes fp, there is no condition that removes fp while keeping fq and fs; the improvement attributed to the proxy could instead come from the fq + fs terms and from the product re-ranking, which is exact when λ = 0 for CIRR. To support the novelty claim, the authors should add controls that replace fp with (a) the query image's own CLIP feature, (b) an image generated from the text layout without query-image conditioning, or (c) a random image feature; if these controls obtain similar gains, the expensive MIGC++ generation step is not the causal ingredient.","section":"§4.4, Table 3, Eq. (1)"},{"comment":"The reported CIRR result is misstated. The abstract and the Introduction both claim a Recall@K of 70.07 at K=10 on CIRR, but Table 1 shows that for LDRE+IP-CIR with ViT-G/14 the Recall@10 is 80.00 and 70.07 is the Recall@5 value. This affects the headline claim and must be corrected.","section":"Abstract and Introduction"},{"comment":"Several implementation details required for reproducibility are missing. The paper states that five proxy images are generated per query but does not specify how their features are ensembled into a single fp (e.g., element-wise averaging, max pooling, or selection). It also does not provide the LLM prompt used to obtain the target captions ft that define the semantic perturbation fs = ft − fo, nor the number of such captions. Without these details, the construction of fRP cannot be reproduced.","section":"§3.3 and §4.1"},{"comment":"No error bars or significance tests are reported. Several improvements are small relative to the variation expected in retrieval metrics, e.g., LinCIR+IP-CIR gains of +0.22 R@1 on CIRR and −0.30 R@10 on FashionIQ Toptee. The authors should report standard deviations over multiple runs or at least over multiple seeds, or justify that the observed differences are stable.","section":"General experimental reporting"}],"minor_comments":[{"comment":"There are typos and inconsistencies, such as 'Implement Details' for 'Implementation Details' and inconsistent use of 'balance metric' versus 'balanced metric'.","section":"Throughout"},{"comment":"The notation max(fp) is ambiguous: it is unclear whether the maximum is taken over the feature dimension or over the entire vector, and the rationale for scaling by the ratio of maxima is not explained.","section":"Eq. (1)"},{"comment":"The appendix restarts figure numbering with 'Figure 1', which conflicts with the main text's Figure 1; renumber the appendix figures.","section":"Appendix and main text figures"},{"comment":"The prompt template for the LLM layout generation is only partially given ('Given an image of {caption}, we show {rule}'); the full instruction, including the format for bbox coordinates and reference modalities, should be included.","section":"§3.2"},{"comment":"The description of the CIRCO test set evaluation is ambiguous; state explicitly whether the official 800-query test split is used and whether the standard exclusion of the query image is applied.","section":"§4.1"},{"comment":"The text says five proxies are generated, but the figure shows only two; clarify that these are example proxy images rather than the full ensemble.","section":"Figure 3"}],"recommendation":"major_revision","confidential_remarks":"The manuscript appears to have a version inconsistency: the abstract reports 70.07@10 on CIRR while Table 1 reports 80.00; the authors should verify all reported numbers. More importantly, the missing control experiments are crucial because the current ablation could be consistent with a much weaker claim that the improvement comes from a combination of existing features and a re-ranking rule rather than from the imagined proxy. Asking for those controls before acceptance would strengthen the paper's contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague —\n\nThe paper's core idea is genuinely new for zero-shot composed image retrieval: instead of projecting the query image into text space, the authors use an LLM to infer a layout, run a controllable generator (MIGC++) conditioned on the query image and relative caption, and use the resulting proxy image's CLIP features to re-rank a text-based baseline. That is a real departure from caption-augmentation methods like CIReVL and LDRE. On three standard benchmarks, the plug-in improves LDRE and LinCIR consistently, and the gain on CIRR with ViT-G/14 is substantial (+3.10 R@1, +2.75 R@10 over LDRE). The ablations are honest: they show that using proxy similarity alone can slightly hurt, and that the gain comes from the full robust feature plus the balance metric.\n\nThe soft spots are real but mostly fixable. The abstract misreports the CIRR result: 70.07 is Recall@K at K=5, not K=10. The balance weight λ is tuned per dataset (0.0 for CIRR, 0.3 for CIRCO, 0.8 for FashionIQ), and the paper admits the method is sensitive to it. Implementation details are missing: the exact LLM prompt, how the five proxy features are ensembled, and how the target captions ft are generated. No code or error bars.\n\nOn the stress-test question: does the imagined proxy actually cause the gain? The main paper's Table 3 does not isolate fp within the robust feature, but the supplementary ablation compares qi+si with and without pi and reports that adding pi improves retrieval. That is an isolation of the proxy contribution, so the concern that the proxy is a no-op is not supported. What is missing is a cheap-control experiment, e.g., replacing fp with the query's own CLIP feature or with a generated image not conditioned on the query; that would show whether the expensive MIGC++ step is necessary. The paper would be stronger with that.\n\nOverall, the central argument holds: generated proxies add useful image-side information, and the method is a solid incremental contribution. It is not a paradigm shift, and the per-dataset tuning tempers the plug-and-play claim. The paper deserves serious refereeing, and I would accept it after revision if the implementation details and the control experiment are added.","headline":"A training-free 'imagined proxy' improves ZS-CIR across three datasets; the core idea is new, the ablations are honest, but the paper needs a cheap-control experiment and implementation details before fully trusting the causal claim.","tokens_in":14905,"tokens_out":2515,"would_cite":false,"duration_ms":22253,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that zero-shot composed image retrieval can be improved without any training by generating an \"imagined proxy\" image—a synthetic picture of the query after the caption's edit—and merging its visual features with the…","keywords":["composed image retrieval","zero-shot retrieval","imagined proxy","training-free method","large language model","controllable image generation","semantic perturbation","retrieval balancing"],"falsifier":"Take a set of query/caption pairs in which the caption alters exactly one attribute (say, color) and check whether the generated proxy preserves all other attributes; if many proxies misapply the edit, and the full method's Recall@1 never beats the text-only baseline for any setting of the balance parameter, the central claim is refuted.","tokens_in":13921,"feed_emoji":"🖼️","tokens_out":8284,"duration_ms":66721,"temperature":0.7,"pith_summary":"The paper tries to establish that zero-shot composed image retrieval—searching for images that match a query image plus a relative caption—can be improved without any training by first imagining what the target looks like. Its proposal, IP-CIR, generates a synthetic \"proxy image\" that applies the caption's edits to the query image, then uses that proxy's visual features to supplement the text features that existing methods rely on. The authors argue that text-only retrieval misses fine-grained visual detail such as color, style, and spatial layout, and they show that composing the proxy feature with the query feature and a text semantic perturbation, plus a balancing metric between text and proxy similarities, lifts several strong baselines on CIRR, CIRCO, and FashionIQ. A sympathetic reader would care because this offers a training-free, plug-and-play route to bring image-side information into a retrieval pipeline that is otherwise text-centric.","feed_headline":"Imagining the target image first improves composed image retrieval","feed_subtitle":"A training-free method merges a generated proxy image with text features, recovering details that text-only retrieval misses.","key_machinery":"The load-bearing mechanism is the imagined proxy and its feature composition. The proxy is a generated image meant to be the query image after the caption's edits, produced by an LLM-reasoned layout (object descriptions, bounding boxes, and per-instance reference modality) rendered by a controllable multi-instance generator that accepts reference-image guidance. The central identity is the robust proxy feature $f_{RP}=f_p+\\frac{\\max(f_p)}{\\max(f_q)}f_q+\\frac{\\max(f_p)}{\\max(f_s)}f_s$, which rescales the query and semantic-perturbation features to the proxy feature's magnitude, plus the balancing similarity $S_f=\\lambda S_t+(1-\\lambda)S_tS_p$, whose product term requires a candidate to score well in both text and proxy evidence. This composition does the work: the ablation in Table 3 shows that raw proxy features alone can slightly hurt CIRR Recall@1 (36.15 to 36.02), while the composed robust feature and balance metric convert the proxy into consistent gains.","core_discovery":"The central claim is that an imagined proxy image carries visual information—instance attributes, style, spatial relationships—that CLIP text features omit, and that this information can be recovered and used for retrieval without any training. IP-CIR first uses an LLM, given a BLIP2 caption of the query image and the relative caption, to generate a layout of objects with bounding boxes and per-instance reference modalities (whether appearance should come from the query image, the text, or both). A controllable generator then renders this layout, using the query image as reference, to produce the proxy. The proxy's CLIP feature $f_p$ is merged with the query feature $f_q$ and a semantic perturbation $f_s=f_t-f_o$ (the difference between LLM-inferred target-caption and original-caption features) into a robust feature $f_{RP}=f_p+\\frac{\\max(f_p)}{\\max(f_q)}f_q+\\frac{\\max(f_p)}{\\max(f_s)}f_s$, and retrieval uses the balanced score $S_f=\\lambda S_t+(1-\\lambda)S_tS_p$, where $S_t$ is the text-side similarity and $S_p$ the proxy-side similarity. As reported in Table 1, this raises CIRR Recall@5 from 66.39 to 70.07 and Recall@10 from 77.25 to 80.00 for LDRE with a ViT-G/14 backbone, and improves CIRCO mAP@10 from 32.24 to 34.26 and FashionIQ average Recall@10 from 45.11 to 45.74.","pith_inferences":["One could extend the same proxy-plus-balance recipe to other relative-to-image retrieval tasks, such as fashion recommendation or video retrieval, since nothing in the pipeline depends on the three benchmarks used.","The sensitivity to λ suggests an automatic, per-query weighting scheme—for instance, estimating proxy reliability from the agreement among the five generated proxies—could remove the main manual tuning step.","Failures on attributes that are hard to generate (logos, pure white backgrounds) point to generation quality rather than fusion as the bottleneck, implying that better controllable generators would improve retrieval even without changing the retrieval method.","The multiplicative balance $S_tS_p$ acts as a soft AND over the two modalities; comparing it with rank-based or learned fusion on the same features would test whether the specific product form is essential or just a convenient choice."],"forward_implications":["Any existing text-based zero-shot CIR method can be wrapped by IP-CIR without retraining; the paper demonstrates improvements when it is added to both LDRE and LinCIR across CLIP ViT-L and ViT-G backbones.","The number of imagined proxies is a practical dial: retrieval accuracy rises with more proxies and the gain saturates, so users can trade compute for accuracy.","The per-dataset balance weight λ must be tuned (0.0 for CIRR, 0.3 for CIRCO, 0.8 for FashionIQ), and the paper notes this reflects how well the generated proxies match each dataset's captions.","Proxy quality is the pivot: when generation is imperfect (e.g., FashionIQ logos and white backgrounds), proxy evidence is less reliable and the method relies on a larger λ, so improving controllable generation should translate directly into retrieval gains."],"supporting_citations":[{"why":"MIGC++ is the controllable generation backbone that renders the imagined proxy from the LLM layout and the query image, so the whole proxy construction depends on it.","marker":"[48]"},{"why":"MIGC provides the multi-instance layout control and reference-image conditioning that MIGC++ builds on for proxy generation.","marker":"[47]"},{"why":"ELITE converts the query image into CLIP-based textual embeddings so the generator can carry the query's instance appearance into the proxy.","marker":"[39]"},{"why":"BLIP2 produces the query image captions that, together with the relative caption, form the LLM's input for layout reasoning.","marker":"[22]"},{"why":"LDRE is the main baseline the method is tested on and the source of the semantic-perturbation idea $f_s=f_t-f_o$ that is merged into the robust proxy feature.","marker":"[43]"},{"why":"LinCIR is the second baseline; combining IP-CIR with it shows the method is not tied to one text-space projection model.","marker":"[17]"},{"why":"Qwen is the LLM used to infer target captions and the proxy layout from the query caption and relative caption.","marker":"[2]"},{"why":"CLIP supplies the shared image-text feature space in which proxy, query, and text features are compared and balanced.","marker":"[31]"}],"fun_headline_variants":["Imagining a proxy image boosts composed image retrieval","Training-free proxy generation improves zero-shot image search","LLM-drafted proxy image sharpens composed retrieval accuracy","Proxy image fills visual gap in composed retrieval"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole method rests on the assumption that the imagined proxy image faithfully keeps the query image's key attributes while applying the caption's changes; if the generated proxy drifts from the query, the merged features inherit that drift and retrieval suffers.","fun_headline_variants_meta":{"raw":{"variants":["Imagining a proxy image boosts composed image retrieval","Training-free proxy generation improves zero-shot image search","LLM-drafted proxy image sharpens composed retrieval accuracy","Proxy image fills visual gap in composed retrieval"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00054,"raw_usage":{"total_tokens":2690,"prompt_tokens":1149,"completion_tokens":1541,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":765,"completion_tokens_details":{"reasoning_tokens":1480}},"tokens_in":765,"tokens_out":1541,"duration_ms":11058,"temperature":1.0,"reasoning_tokens":1480,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T13:58:32.782041+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a set of query/caption pairs in which the caption alters exactly one attribute (say, color) and check whether the generated proxy preserves all other attributes; if many proxies misapply the edit, and the full method's Recall@1 never beats the text-only baseline for any setting of the balance parameter, the central claim is refuted.","supporting_citations":[{"cited_title":"Migc: Multi-instance generation controller for text-to-image synthesis","cited_arxiv_id":null,"evidence_quote":"MIGC provides the multi-instance layout control and reference-image conditioning that MIGC++ builds on for proxy generation."},{"cited_title":"Ldre: Llm-based divergent reasoning and ensemble for zero-shot composed image re- trieval","cited_arxiv_id":null,"evidence_quote":"LDRE is the main baseline the method is tested on and the source of the semantic-perturbation idea $f_s=f_t-f_o$ that is merged into the robust proxy feature."},{"cited_title":"Language-only training of zero- shot composed image retrieval","cited_arxiv_id":null,"evidence_quote":"LinCIR is the second baseline; combining IP-CIR with it shows the method is not tied to one text-space projection model."},{"cited_title":"Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever","cited_arxiv_id":null,"evidence_quote":"CLIP supplies the shared image-text feature space in which proxy, query, and text features are compared and balanced."}],"review_version":1}