{"id":"0dee574b-6979-4fe9-9483-c652f4038739","arxiv_id":"2501.15379","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"DAR augments text queries with images generated by Stable Diffusion to improve zero-shot interactive image retrieval, at the cost of inference-time generation.","lead":"This paper introduces DAR, a system that uses a language model and a diffusion model to generate text and image representations of a conversational query, then retrieves matching photos without any finetuning. It reports that on complex multi-turn queries DAR can beat finetuned retrieval models on Hits@10, though the specific gain from the diffusion step is not isolated.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No ablation isolates diffusion: the DAR_zs gain over BLIP_zs may come entirely from R1 query reformulation, not from generated images. The missing K=0 control is the key gap.","rationale":"The reader's weakest_assumption identifies exactly the concern I would prioritize: the paper never ablates the diffusion component while keeping the LLM reformulation. I checked the experimental sections to see whether any reported comparison could serve as a K=0 control. Section 4.2 compares DAR_zs to BLIP_zs, but those conditions differ by both R1 and the generated images, so the comparison cannot isolate diffusion. Section 5.3 varies the number of generated images from one to three, but all those conditions still include generated images, and the reference point is ChatIR, not a no-image DAR. Thus the central contribution named in the title is not directly tested. This is a genuine experimental gap, not a disagreement with the community consensus or an internal inconsistency. The paper has real strengths: the system is clearly described, the code is available, and the full DAR pipeline consistently improves over several baselines across four benchmarks. Those strengths support a conditional acceptance rather than a rejection. I would not move the reader's verdict; the same missing ablation that justified CONDITIONAL remains the decisive issue. The proposed K=0 test is cheap and would settle the attribution question directly.","tokens_in":15564,"tokens_out":3571,"duration_ms":33393,"concrete_test":"Run Algorithm 1 on FLAN_BLIP2 and VisDial with K=0: skip the DiffusionModel call (line 11), keep R1 reformulation, set alpha=1 and beta=0 in Eq. (4), and keep all other settings, prompts, and QA turns identical to DAR_zs. Compare per-turn Hits@10 against the reported K=3 DAR_zs using the same random seed and, ideally, several seeds for variance estimates. If turn-10 Hits@10 with K=0 is within a couple of points of K=3, the diffusion images are not load-bearing; if K=0 drops materially toward BLIP_zs, the diffusion component is supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that diffusion-generated images are what make DAR work. Eq. (4) fuses E_t(S_t) with the E_v(image) terms, and Algorithm 1 always calls the diffusion model before retrieval. However, no experiment runs the pipeline with K=0 (images disabled) while keeping the R1 reformulation. The strongest reported result, DAR_zs vs BLIP_zs +7.61% Hits@10 on FLAN_BLIP2 (Section 4.2), is not a controlled comparison of diffusion: BLIP_zs receives the raw dialogue, whereas DAR_zs receives R1's reformulated query plus generated images. Section 5.3 only varies K between 1 and 3 and reports improvements relative to the ChatIR baseline, never relative to a K=0 DAR variant, so the marginal contribution of the generated images is never shown. If R1 alone (alpha=1, beta=0) already produces most of the gain, the 'diffusion augmented' mechanism and the many-to-one mapping framing (Section 3.3) would not be supported. Secondary issues (alpha/beta/K selected on the evaluation benchmarks, no error bars, and the abstract attributing the 7.61% gain to 'finetuned I-TIR models' when the comparison is against BLIP_zs) compound the uncertainty but are less load-bearing than the missing K=0 condition.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Diffusion Augmented Retrieval (DAR), a training-free framework for interactive text-to-image retrieval. At each turn it uses an LLM to reformulate the dialogue (R1), generates K synthetic images with Stable Diffusion from multiple prompts (R2), encodes the reformulated text and generated images with a frozen MLLM, and fuses these embeddings via Eq. (4) before ranking images in the corpus. Experiments on four benchmarks compare DAR variants built on BLIP, ChatIR, and COCOFT encoders, reporting consistent Hits@10 gains over the corresponding baselines, with the largest gain (7.61%) on FLAN_BLIP2 after ten turns. The paper also includes compatibility tests with CLIP/BEiT-3 and Stable Diffusion v2-1, a small K-sensitivity study, and a qualitative analysis of generated images.","tokens_in":15849,"tokens_out":3000,"duration_ms":26861,"significance":"If the attribution of the gains to the diffusion component is established, DAR is a practically attractive result: it avoids finetuning MLLMs, relies on off-the-shelf components, shows consistent improvements across four benchmarks, and releases code. The compatibility analysis across encoders and diffusion backbones is a useful engineering contribution. However, the central scientific claim—that diffusion-generated images are what drive the improvement—is not yet supported, because no ablation removes the diffusion component while retaining the LLM reformulation. In addition, the hyperparameters alpha, beta, and K are tuned on the same benchmarks used for reporting, and no error bars or significance tests are given. These gaps are fixable but are load-bearing for the paper's framing.","major_comments":[{"comment":"The DAR_zs versus BLIP_zs comparison does not control for the LLM reformulation. DAR_zs applies R1 plus K generated images, while BLIP_zs scores the raw dialogue; the 7.61% gain in FIG 3(d) could be entirely due to R1. A K=0 ablation (keep R1, set beta=0 in Eq. (4)) is missing. Section 5.3 varies K from 1 to 3 only relative to the ChatIR baseline and never reports a K=0 DAR variant, so the marginal contribution of the diffusion-generated images is never isolated. Please add a K=0 condition for DAR_zs (and ideally DAR_chatir) on all four benchmarks.","section":"§4.2, Eq. (4), Algorithm 1"},{"comment":"The weighting factors alpha and beta (0.7/0.3 then 0.5/0.5) and the number of generated images K=3 are selected empirically on the same four benchmarks used for the reported results, with no validation split or nested procedure described. Section 5.3 further reports that K=3 gives the best FLAN_BLIP2 result, which makes the headline numbers partially optimized on the test sets. Please either use a held-out validation set for selection, report full sensitivity over alpha/beta/K with clear train/test separation, or include error bars over dialogue subsets or random seeds.","section":"§4.1 'Experimental Settings' and §5.3"},{"comment":"The abstract says DAR 'surpasses finetuned I-TIR models by up to 7.61% in Hits@10 after ten turns,' but the 7.61% figure is the DAR_zs improvement over BLIP_zs, which is not a finetuned model; the gain over the finetuned ChatIR baseline is 4.22% (Section 4.4). This conflates the two comparisons and should be corrected, with the baseline names stated explicitly wherever percentages are quoted.","section":"Abstract and §§4.2/4.4"}],"minor_comments":[{"comment":"The caption contains the fragment 'reduction in performance' and states that Hits@10 is cumulative with early stopping ('we cease to use additional dialogues once the image attains a top-k rank'); please define the metric precisely and state whether all baselines use the same stopping rule.","section":"Figure 3 caption"},{"comment":"The phrase 'the most challenging benchmark ... where the best-performing model achieves the lowest Hits@10 score' is circular; clarify that FLAN_BLIP2 is singled out because all baselines plateau there, not because of the best model's absolute score.","section":"§4.3"},{"comment":"Typo: 'an balance' should be 'a balance' in the sentence about the default configuration.","section":"§5.3"},{"comment":"The qualitative analysis in Figure 4 is illustrative, but the claim that generated images 'become increasingly photorealistic' is not quantified; consider adding a small scoring or consistency analysis rather than only example images.","section":"§5.1"}],"recommendation":"major_revision","confidential_remarks":"The paper fits the SIGIR scope and the system-level results are plausible, but the novelty hinges on the diffusion component: without a K=0 ablation, the contribution reduces to LLM-based dialogue reformulation plus simple embedding averaging, which is considerably more incremental. The hyperparameter selection on the test benchmarks and the abstract's conflation of the 7.61% figure with finetuned baselines also need correction. I encourage the authors to address these with additional experiments; the framework itself is promising and the writing is generally clear."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea here is genuinely nice: reformulate a multi-turn dialogue with an LLM, generate a few candidate images with Stable Diffusion, fuse the text and image embeddings, and retrieve by cosine similarity. For interactive text-to-image retrieval, that specific combination is new, and the paper ships code and experiments on four benchmarks. The consistent improvements over the zero-shot BLIP baseline and the out-of-distribution robustness on FLAN_BLIP2 are worth taking seriously.\n\nBut the paper has one load-bearing gap: it never ablates the diffusion component. DAR_zs always runs R1 reformulation plus generated images, while BLIP_zs gets the raw dialogue. The gain could come entirely from R1's rewritten query. Section 5.3 only varies K from 1 to 3 compared against ChatIR, never against DAR with K=0. That's the missing control that would isolate the diffusion contribution. Without it, the phrase \"diffusion augmented\" is not really supported.\n\nThe other issues are secondary but real. Alpha, beta, and K are tuned on the same benchmarks used for reporting final numbers, with no validation split and no error bars or significance tests. And the abstract says DAR surpasses finetuned I-TIR models by 7.61%, but that number is DAR_zs versus BLIP_zs (zero-shot BLIP, not finetuned). The actual finetuned comparison is the 4.22% over ChatIR. That should be fixed.\n\nAll of this is addressable. The K=0 ablation would take an afternoon given the code, and the hyperparameter disclosure can be clarified. The paper is coherent, the writing is clear, and the authors are honest about the inference overhead. I don't see any attempt to hide the missing ablation—it seems like an oversight rather than a deliberate move.\n\nFor a researcher working on conversational retrieval or using generative models for query augmentation, this is a useful paper to know. The framework is simple enough to build on, and the negative results about finetuned models are thought-provoking. I would send it to a serious referee: the idea is publishable, the gap is identifiable, and a revision that isolates the diffusion contribution would make it much stronger.","headline":"The paper presents a clean training-free I-TIR recipe with public code, but it never runs the key control: a K=0 variant without diffusion images, so the central claim that generated images drive the gains is not actually demonstrated.","tokens_in":16353,"tokens_out":2480,"would_cite":true,"duration_ms":23653,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that an interactive text-to-image retrieval system can match or beat finetuned multimodal models without any training by using a diffusion model to generate several images of the user's query and fusing their embeddings…","keywords":["Interactive Text-to-image Retrieval","Conversational IR","Diffusion Augmented Retrieval","diffusion models","zero-shot retrieval","multimodal large language models","feature fusion","Hits@10"],"falsifier":"Run DAR on FLAN_BLIP2 with zero generated images (K=0, so the fused representation is just the R1-refined text embedding) and compare ten-turn Hits@10 with the reported K=3 result. If K=0 matches or beats K=3, the diffusion component is not what carries the improvement; if K=0 drops sharply, the central claim is supported.","tokens_in":15368,"feed_emoji":"🖼️","tokens_out":8943,"duration_ms":68961,"temperature":0.7,"pith_summary":"Interactive text-to-image retrieval (I-TIR) currently relies on finetuning multimodal large language models, a step that costs large GPU budgets and narrows the models' pretrained knowledge. The paper's claim is that this finetuning can be skipped: a frozen encoder retrieves well if a diffusion model first generates several images of what the dialogue is describing, and the system scores candidates against a fused embedding of the refined text and those synthetic images. On four benchmarks, this training-free approach, called Diffusion Augmented Retrieval (DAR), matches finetuned models on the first, simple query and overtakes them as dialogues grow longer, reaching a 7.61% higher Hits@10 (share of targets in the top ten) after ten turns on the most out-of-distribution benchmark. If true, this would let smaller teams deploy and update interactive retrieval systems without expensive finetuning, while preserving the generalisation that pretrained models already have.","feed_headline":"No finetuning: generated images beat finetuned retrieval by 7.61%","feed_subtitle":"Frozen encoders plus three generated images per turn beat fine-tuned interactive retrieval by up to 7.61%.","key_machinery":"The load-bearing object is the fused multi-faceted query embedding $F_t$ of Eq. (4): a weighted sum of the text embedding of the LLM-refined dialogue and the summed image embeddings of $K$ diffusion-generated images, with weights $\\alpha+\\beta=1$ set to $\\alpha=0.7,\\beta=0.3$ for the first two turns and $0.5/0.5$ afterwards. It carries the argument by injecting the diffusion model's learned text-to-image prior into the frozen encoder's embedding space, converting a one-to-one text-to-image match into a many-to-one match. The two reformulation pipelines (R1 for the retrieval query and R2 for the diffusion prompts) keep both the encoder and the generator inside their pretrained input distributions, which is what makes the whole chain training-free.","core_discovery":"DAR's central discovery is that a diffusion model's pretrained text-to-image mapping can substitute for retrieval finetuning. At each turn $t$, an LLM rewrites the dialogue into a retrieval-friendly query $S_t$ (pipeline R1) and into $K$ diverse diffusion prompts (pipeline R2); the diffusion model generates images $\\hat{I}_{t,k}$, and the query representation becomes $F_t = \\alpha E(S_t) + \\beta \\sum_{k=1}^{K} E(\\hat{I}_{t,k})$ with $\\alpha+\\beta=1$, after which candidate images are ranked by cosine similarity to $F_t$. This many-to-one representation—several imagined images plus the text—lets a frozen BLIP encoder beat a model finetuned on 123k VisDial dialogues by 4.22% Hits@10 at turn 10 on the hardest benchmark, and beat its unfinetuned counterpart by 7.61%. The authors' stated explanation is that finetuning compresses the pretrained distribution, whereas the generated images expand the query representation to cover complex, out-of-distribution dialogue.","pith_inferences":["A K=0 ablation that keeps R1 but drops the generated images would separate the reformulation effect from the diffusion effect; the paper leaves that comparison open.","The same fusion recipe could transfer to composed image retrieval or video retrieval, where a frozen dual encoder plus generated visual candidates may behave like zero-shot query expansion.","Because image generation costs about 5 seconds per turn versus 0.5 seconds for reformulation, reusing or caching generated images across turns is a natural latency-saving extension.","The turn-dependent weights (0.7/0.3 then 0.5/0.5) are fixed empirically; making alpha and beta adaptive to per-turn uncertainty is a testable refinement."],"forward_implications":["On the most complex benchmark (FLAN_BLIP2), the frozen DAR_zs reaches 7.61% higher Hits@10 than the zero-shot BLIP baseline at turn 10, and the gap grows with each added dialogue turn.","Combining DAR with an already finetuned encoder (DAR_chatir) improves on that encoder by 2.42% Hits@10 on VisDial and by up to 9.4% on the other benchmarks, without any further training.","Even one generated image per turn provides most of the benefit (6.43% over ChatIR on FLAN_BLIP2), and three images give 7.61%, after which returns diminish.","The framework is compatible with CLIP, BLIP, and BEiT-3 encoders and with at least two Stable Diffusion versions, so the gains are not tied to one model pair.","Finetuning on a single-turn dataset (MSCOCO) helps only in early turns and vanishes by turn 8, indicating that single-turn retrieval finetuning cannot replace multi-faceted generation."],"supporting_citations":[{"why":"Supplies the three dialogue-based I-TIR datasets and the finetuned ChatIR retrieval model that DAR is measured against.","marker":"[13]"},{"why":"Provides BLIP, the default frozen image-text encoder whose embeddings are fused in Eq. (4).","marker":"[14]"},{"why":"Stable Diffusion 3, the default diffusion generator whose text-to-image prior produces the imagined images.","marker":"[4]"},{"why":"Stable Diffusion v2-1, the alternative generator used to show DAR is not tied to one diffusion model.","marker":"[24]"},{"why":"BLIP-3/xGen-MM, the LLM that carries out both the R1 dialogue reformulation and R2 diffusion-prompt generation.","marker":"[34]"},{"why":"VisDial, the benchmark on which ChatIR was finetuned and on which DAR remains competitive.","marker":"[3]"},{"why":"MSCOCO, used for the COCOFT baseline that isolates the value of single-turn retrieval finetuning.","marker":"[16]"}],"fun_headline_variants":["Diffusion-generated images beat finetuned retrieval by 7.61%","Training-free diffusion tops finetuned retrieval by 7.61%","No finetuning: diffusion images improve retrieval by 7.61%","Frozen encoder + diffusion images outmatch finetuned by 7.61%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the diffusion-generated images, rather than the LLM dialogue reformulation alone, cause the measured gains, and the paper does not report the K=0 ablation that would isolate this.","fun_headline_variants_meta":{"raw":{"variants":["Diffusion-generated images beat finetuned retrieval by 7.61%","Training-free diffusion tops finetuned retrieval by 7.61%","No finetuning: diffusion images improve retrieval by 7.61%","Frozen encoder + diffusion images outmatch finetuned by 7.61%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001421,"raw_usage":{"total_tokens":5795,"prompt_tokens":1067,"completion_tokens":4728,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":683,"completion_tokens_details":{"reasoning_tokens":4644}},"tokens_in":683,"tokens_out":4728,"duration_ms":29462,"temperature":1.0,"reasoning_tokens":4644,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T14:20:15.033608+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run DAR on FLAN_BLIP2 with zero generated images (K=0, so the fused representation is just the R1-refined text embedding) and compare ten-turn Hits@10 with the reported K=3 result. If K=0 matches or beats K=3, the diffusion component is not what carries the improvement; if K=0 drops sharply, the central claim is supported.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the three dialogue-based I-TIR datasets and the finetuned ChatIR retrieval model that DAR is measured against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides BLIP, the default frozen image-text encoder whose embeddings are fused in Eq. (4)."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Stable Diffusion 3, the default diffusion generator whose text-to-image prior produces the imagined images."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Stable Diffusion v2-1, the alternative generator used to show DAR is not tied to one diffusion model."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"VisDial, the benchmark on which ChatIR was finetuned and on which DAR remains competitive."},{"cited_title":"Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C","cited_arxiv_id":null,"evidence_quote":"MSCOCO, used for the COCOFT baseline that isolates the value of single-turn retrieval finetuning."}],"review_version":1}