Pith. sign in

REVIEW 3 major objections 4 minor 43 references

Diffusion Augmented Retrieval: A Training-Free Approach to Interactive Text-to-Image Retrieval

T0 review · 3 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that an interactive text-to-image retrieval system can match or beat finetuned multimodal models without any training by using a diffusion model to generate several images of the user's query and fusing their embeddings…

desk verdict The paper presents a clean training-free I-TIR recipe with public code, but it never runs the key control: a K=0 variant without diffusion images, so the central claim that generated images drive the gains is not actually demonstrated. read the letter →

arxiv 2501.15379 v2 pith:7PYUZTIH submitted 2025-01-26 cs.IR cs.AIcs.CV

Zijun Long , Kangheng Liang , Gerardo Aragon-Camarasa , Richard Mccreadie , Paul Henderson This is my paper · ORCID
classification cs.IRcs.AIcs.CV
keywords InteractiveText-to-imageRetrievalConversationalIRDiffusionAugmentedmodelszero-shotmultimodallargelanguagefeaturefusionHits@10
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Interactive text-to-image retrieval (I-TIR) currently relies on finetuning multimodal large language models, a step that costs large GPU budgets and narrows the models' pretrained knowledge. The paper's claim is that this finetuning can be skipped: a frozen encoder retrieves well if a diffusion model first generates several images of what the dialogue is describing, and the system scores candidates against a fused embedding of the refined text and those synthetic images. On four benchmarks, this training-free approach, called Diffusion Augmented Retrieval (DAR), matches finetuned models on the first, simple query and overtakes them as dialogues grow longer, reaching a 7.61% higher Hits@10 (share of targets in the top ten) after ten turns on the most out-of-distribution benchmark. If true, this would let smaller teams deploy and update interactive retrieval systems without expensive finetuning, while preserving the generalisation that pretrained models already have.

What carries the argument

The load-bearing object is the fused multi-faceted query embedding $F_t$ of Eq. (4): a weighted sum of the text embedding of the LLM-refined dialogue and the summed image embeddings of $K$ diffusion-generated images, with weights $\alpha+\beta=1$ set to $\alpha=0.7,\beta=0.3$ for the first two turns and $0.5/0.5$ afterwards. It carries the argument by injecting the diffusion model's learned text-to-image prior into the frozen encoder's embedding space, converting a one-to-one text-to-image match into a many-to-one match. The two reformulation pipelines (R1 for the retrieval query and R2 for the diffusion prompts) keep both the encoder and the generator inside their pretrained input distributions, which is what makes the whole chain training-free.

What would settle it

Run DAR on FLAN_BLIP2 with zero generated images (K=0, so the fused representation is just the R1-refined text embedding) and compare ten-turn Hits@10 with the reported K=3 result. If K=0 matches or beats K=3, the diffusion component is not what carries the improvement; if K=0 drops sharply, the central claim is supported.

Watch

Extended reading notes

Core claim

DAR's central discovery is that a diffusion model's pretrained text-to-image mapping can substitute for retrieval finetuning. At each turn $t$, an LLM rewrites the dialogue into a retrieval-friendly query $S_t$ (pipeline R1) and into $K$ diverse diffusion prompts (pipeline R2); the diffusion model generates images $\hat{I}_{t,k}$, and the query representation becomes $F_t = \alpha E(S_t) + \beta \sum_{k=1}^{K} E(\hat{I}_{t,k})$ with $\alpha+\beta=1$, after which candidate images are ranked by cosine similarity to $F_t$. This many-to-one representation—several imagined images plus the text—lets a frozen BLIP encoder beat a model finetuned on 123k VisDial dialogues by 4.22% Hits@10 at turn 10 on the hardest benchmark, and beat its unfinetuned counterpart by 7.61%. The authors' stated explanation is that finetuning compresses the pretrained distribution, whereas the generated images expand the query representation to cover complex, out-of-distribution dialogue.

Load-bearing premise

The load-bearing premise is that the diffusion-generated images, rather than the LLM dialogue reformulation alone, cause the measured gains, and the paper does not report the K=0 ablation that would isolate this.

Editorial extensions

If this is right

  • On the most complex benchmark (FLAN_BLIP2), the frozen DAR_zs reaches 7.61% higher Hits@10 than the zero-shot BLIP baseline at turn 10, and the gap grows with each added dialogue turn.
  • Combining DAR with an already finetuned encoder (DAR_chatir) improves on that encoder by 2.42% Hits@10 on VisDial and by up to 9.4% on the other benchmarks, without any further training.
  • Even one generated image per turn provides most of the benefit (6.43% over ChatIR on FLAN_BLIP2), and three images give 7.61%, after which returns diminish.
  • The framework is compatible with CLIP, BLIP, and BEiT-3 encoders and with at least two Stable Diffusion versions, so the gains are not tied to one model pair.
  • Finetuning on a single-turn dataset (MSCOCO) helps only in early turns and vanishes by turn 8, indicating that single-turn retrieval finetuning cannot replace multi-faceted generation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A K=0 ablation that keeps R1 but drops the generated images would separate the reformulation effect from the diffusion effect; the paper leaves that comparison open.
  • The same fusion recipe could transfer to composed image retrieval or video retrieval, where a frozen dual encoder plus generated visual candidates may behave like zero-shot query expansion.
  • Because image generation costs about 5 seconds per turn versus 0.5 seconds for reformulation, reusing or caching generated images across turns is a natural latency-saving extension.
  • The turn-dependent weights (0.7/0.3 then 0.5/0.5) are fixed empirically; making alpha and beta adaptive to per-turn uncertainty is a testable refinement.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes Diffusion Augmented Retrieval (DAR), a training-free framework for interactive text-to-image retrieval. At each turn it uses an LLM to reformulate the dialogue (R1), generates K synthetic images with Stable Diffusion from multiple prompts (R2), encodes the reformulated text and generated images with a frozen MLLM, and fuses these embeddings via Eq. (4) before ranking images in the corpus. Experiments on four benchmarks compare DAR variants built on BLIP, ChatIR, and COCOFT encoders, reporting consistent Hits@10 gains over the corresponding baselines, with the largest gain (7.61%) on FLAN_BLIP2 after ten turns. The paper also includes compatibility tests with CLIP/BEiT-3 and Stable Diffusion v2-1, a small K-sensitivity study, and a qualitative analysis of generated images.

Significance. If the attribution of the gains to the diffusion component is established, DAR is a practically attractive result: it avoids finetuning MLLMs, relies on off-the-shelf components, shows consistent improvements across four benchmarks, and releases code. The compatibility analysis across encoders and diffusion backbones is a useful engineering contribution. However, the central scientific claim—that diffusion-generated images are what drive the improvement—is not yet supported, because no ablation removes the diffusion component while retaining the LLM reformulation. In addition, the hyperparameters alpha, beta, and K are tuned on the same benchmarks used for reporting, and no error bars or significance tests are given. These gaps are fixable but are load-bearing for the paper's framing.

major comments (3)
  1. [§4.2, Eq. (4), Algorithm 1] The DAR_zs versus BLIP_zs comparison does not control for the LLM reformulation. DAR_zs applies R1 plus K generated images, while BLIP_zs scores the raw dialogue; the 7.61% gain in FIG 3(d) could be entirely due to R1. A K=0 ablation (keep R1, set beta=0 in Eq. (4)) is missing. Section 5.3 varies K from 1 to 3 only relative to the ChatIR baseline and never reports a K=0 DAR variant, so the marginal contribution of the diffusion-generated images is never isolated. Please add a K=0 condition for DAR_zs (and ideally DAR_chatir) on all four benchmarks.
  2. [§4.1 'Experimental Settings' and §5.3] The weighting factors alpha and beta (0.7/0.3 then 0.5/0.5) and the number of generated images K=3 are selected empirically on the same four benchmarks used for the reported results, with no validation split or nested procedure described. Section 5.3 further reports that K=3 gives the best FLAN_BLIP2 result, which makes the headline numbers partially optimized on the test sets. Please either use a held-out validation set for selection, report full sensitivity over alpha/beta/K with clear train/test separation, or include error bars over dialogue subsets or random seeds.
  3. [Abstract and §§4.2/4.4] The abstract says DAR 'surpasses finetuned I-TIR models by up to 7.61% in Hits@10 after ten turns,' but the 7.61% figure is the DAR_zs improvement over BLIP_zs, which is not a finetuned model; the gain over the finetuned ChatIR baseline is 4.22% (Section 4.4). This conflates the two comparisons and should be corrected, with the baseline names stated explicitly wherever percentages are quoted.
minor comments (4)
  1. [Figure 3 caption] The caption contains the fragment 'reduction in performance' and states that Hits@10 is cumulative with early stopping ('we cease to use additional dialogues once the image attains a top-k rank'); please define the metric precisely and state whether all baselines use the same stopping rule.
  2. [§4.3] The phrase 'the most challenging benchmark ... where the best-performing model achieves the lowest Hits@10 score' is circular; clarify that FLAN_BLIP2 is singled out because all baselines plateau there, not because of the best model's absolute score.
  3. [§5.3] Typo: 'an balance' should be 'a balance' in the sentence about the default configuration.
  4. [§5.1] The qualitative analysis in Figure 4 is illustrative, but the claim that generated images 'become increasingly photorealistic' is not quantified; consider adding a small scoring or consistency analysis rather than only example images.

Circularity Check

0 steps flagged · score 0.0 of 10

No construction-level circularity: DAR is an empirical composition of independently pretrained LLM, diffusion, and encoder components, and no equation or cited result reduces the headline performance to the paper's own inputs.

full rationale

DAR is presented as an empirical pipeline (reformulation S_t = R1(C_t), prompt generation P_{t,k} = R2(S_t, k), diffusion-based image generation, and feature fusion F_t = alpha*E_t(S_t) + beta*sum_k E_v(I_hat_{t,k})), not as a derivation from first principles. Each component is a pretrained external model (BLIP-3, Stable Diffusion 3, BLIP encoder), and the reported Hits@10 values depend on encoding and ranking external benchmark images, so the central result is not equivalent to the method's definition by construction. The paper does not invoke a self-citation chain or a uniqueness theorem; the few self-citations in the references are related-work items and are not load-bearing premises. Two legitimate concerns, namely the absence of a K=0 ablation isolating the diffusion contribution from the R1 reformulation and the manual setting of alpha, beta, and K on the evaluation benchmarks, are experimental validity issues rather than circular derivation, since no fitted parameter is renamed as an independent prediction and no equation reduces to its own input.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The system introduces no new entities; it composes existing pretrained models. The main burdens are an untested assumption about the diffusion component's causal contribution, and empirical hyperparameter choices made on the evaluation benchmarks.

free parameters (2)
  • alpha (text fusion weight) = 0.7 for turns 1-2, 0.5 from turn 3
    Set empirically in Section 4.1 with no validation split, so it is tuned on the same benchmarks used for evaluation.
  • K (number of diffusion-generated images per turn) = 3
    Chosen in Section 5.3 after observing diminishing returns, a post hoc selection on the same benchmarks.
assumptions (3)
  • domain assumption Stable Diffusion can generate images that accurately reflect the user's intent from the reformulated prompts.
    The entire DAR approach relies on generated images being semantically faithful to the dialogue (Section 3.3). If generation is noisy or off-target, the fused embedding is degraded.
  • domain assumption BLIP embeddings of synthetic images are comparable to embeddings of real candidate images in the same space.
    Feature fusion in Eq. (4) adds generated-image embeddings to text embeddings and matches against real image embeddings (Section 3.4.1). No calibration between synthetic and real image embedding distributions is provided.
  • ad hoc to paper The performance gain attributed to diffusion is not solely due to the LLM-based dialogue reformulation.
    The paper asserts diffusion provides the benefit, but no ablation with K=0 is reported (Sections 4.2 and 5.3). This is an untested assumption central to the claim.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Diffusion Augmented Retrieval: A Training-Free Approach to Interactive Text-to-Image Retrieval." pith.science (2026). https://pith.science/paper/7PYUZTIH

@misc{pith2026250115379,
  author       = {Pith},
  title        = {Pith review of: Diffusion Augmented Retrieval: A Training-Free Approach to Interactive Text-to-Image Retrieval},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7PYUZTIH}},
  note         = {Machine review of arXiv:2501.15379}
}
read the original abstract

Interactive Text-to-image retrieval (I-TIR) is an important enabler for a wide range of state-of-the-art services in domains such as e-commerce and education. However, current methods rely on finetuned Multimodal Large Language Models (MLLMs), which are costly to train and update, and exhibit poor generalizability. This latter issue is of particular concern, as: 1) finetuning narrows the pretrained distribution of MLLMs, thereby reducing generalizability; and 2) I-TIR introduces increasing query diversity and complexity. As a result, I-TIR solutions are highly likely to encounter queries and images not well represented in any training dataset. To address this, we propose leveraging Diffusion Models (DMs) for text-to-image mapping, to avoid finetuning MLLMs while preserving robust performance on complex queries. Specifically, we introduce Diffusion Augmented Retrieval (DAR), a framework that generates multiple intermediate representations via LLM-based dialogue refinements and DMs, producing a richer depiction of the user's information needs. This augmented representation facilitates more accurate identification of semantically and visually related images. Extensive experiments on four benchmarks show that for simple queries, DAR achieves results on par with finetuned I-TIR models, yet without incurring their tuning overhead. Moreover, as queries become more complex through additional conversational turns, DAR surpasses finetuned I-TIR models by up to 7.61% in Hits@10 after ten turns, illustrating its improved generalization for more intricate queries.

Figures

Figures reproduced from arXiv: 2501.15379 by the authors.

Figure 1
Figure 1. The core idea underpinning DAR is to produce multiple [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The overall architecture of our proposed framework DAR. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. This is because these pairs are initialized from the same pre-trained weights, ensuring a fair comparison that shows how the addition of DAR affects performance. 4.2 Zero-Shot I-TIR Performance We first investigate how our proposed DAR framework performs under zero-shot I-TIR conditions. Specifically, BLIP_zs denotes the baseline where BLIP is used with its original, publicly available pretrained weights—without any… view at source ↗
Figures from the paper (2 more)
Figure 3
Figure 3. Figure 3: The experimental results for four evaluated benchmarks. Note that for Hits@10, a higher value is better. It is an [PITH_FULL_IMAGE:figures/full_fig_p007_3.png]
Figure 4
Figure 4. Figure 4: The examples of generated images in DAR. [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

43 extracted references · 35 canonical work pages

  1. [1]

    Antonia Creswell, Tom White, Vincent Dumoulin, Kai Arulkumaran, Biswa Sen- gupta, and Anil A Bharath. 2018. Generative adversarial networks: An overview. IEEE signal processing magazine 35, 1 (2018), 53–65

    work page 2018

  2. [2]

    Florinel-Alin Croitoru, Vlad Hondru, Radu Tudor Ionescu, and Mubarak Shah

  3. [3]

    Abhishek Das, Satwik Kottur, Khushi Gupta, Avi Singh, Deshraj Yadav, Stefan Lee, José M. F. Moura, Devi Parikh, and Dhruv Batra. 2019. Visual Dialog. IEEE Trans. Pattern Anal. Mach. Intell. 41, 5 (2019), 1242–1256

    work page 2019

  4. [4]

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion English, and Robin Rombach. 2024. Scaling Rectified Flow Transformers for High-Resolution Image Synthesis. In Proceedings of the International Conference on Machine Learning, ICML 2024

    work page 2024

  5. [5]

    Xuri Ge, Fuhai Chen, et al . 2021. Structured multi-modal feature embedding and alignment for image-sentence retrieval. In Proceedings of the 29th ACM international conference on multimedia . 5185–5193

    work page 2021

  6. [6]

    Xuri Ge, Songpei Xu, et al. 2024. 3SHNet: Boosting image–sentence retrieval via visual semantic–spatial self-highlighting. Information Processing & Management 61, 4 (2024), 103716

    work page 2024

  7. [7]

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. In Proceedings of the Advances in neural information processing systems conference, NeurIPS 2014, Vol. 27

    work page 2014

  8. [8]

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2020. Generative adversarial networks. Commun. ACM 63, 11 (2020), 139–144

    2020

Show all 43 references
  1. [9]

    Jie Gui, Zhenan Sun, Yonggang Wen, Dacheng Tao, and Jieping Ye. 2020. A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications. CoRR abs/2001.06937 (2020)

    2020 arXiv

  2. [10]

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising diffusion probabilistic models. In Proceedings of the Advances in neural information processing systems conference, NeurIPS, Vol. 33. 6840–6851

    2020

  3. [11]

    Ankit Kumar, Richa Sharma, and Punam Bedi. 2024. Towards Optimal NLP Solutions: Analyzing GPT and LLaMA-2 Models Across Model Scale, Dataset Size, and Task Diversity. Engineering, Technology & Applied Science Research 14, 3 (2024), 14219–14224

    2024

  4. [12]

    Saehyung Lee, Sangwon Yu, Junsung Park, Jihun Yi, and Sungroh Yoon. 2024. In- teractive Text-to-Image Retrieval with Large Language Models: A Plug-and-Play Approach. In Proceedings of the Annual Meeting of the Association for Computa- tional Linguistics, ACL 2024. Association ...

    2024

  5. [13]

    Matan Levy, Rami Ben-Ari, Nir Darshan, and Dani Lischinski. 2023. Chatting Makes Perfect: Chat-based Image Retrieval. In Proceedings of the Advances in Neural Information Processing Systems, NeurIPS, 2023

    2023

  6. [14]

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven C. H. Hoi. 2022. BLIP: Boot- strapping Language-Image Pre-training for Unified Vision-Language Understand- ing and Generation. In Proceedings of the International Conference on Machine Learning, ICML 2022, Vol. 162. PMLR, 12888–12900

    2022

  7. [15]

    Yongqi Li, Wenjie Wang, Leigang Qu, Liqiang Nie, Wenjie Li, and Tat-Seng Chua

  8. [16]

    Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C

    Tsung-Yi Lin, Michael Maire, Serge J. Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. 2014. Microsoft COCO: Common Objects in Context. In Proceedings of The European Conference on Computer Vision ECCV 2014, Vol. 8693. Springer, 740–755

    2014

  9. [17]

    Zijun Long, Xuri Ge, et al . 2024. Cfir: Fast and effective long-text to image retrieval for large corpora. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval . 2188–2198

    2024

  10. [18]

    Zijun Long, George Killick, Richard McCreadie, and Gerardo Aragon Camarasa

  11. [19]

    Zijun Long, Lipeng Zhuang, et al. 2024. Understanding and Mitigating Human- Labelling Errors in Supervised Contrastive Learning. In European Conference on Computer Vision. Springer, 435–454

    2024

  12. [20]

    Zijun Long, Lipeng Zhuang, George Killick, Zaiqiao Meng, Richard Mccreadie, and Gerardo Aragon-Camarasa. 2024. Clce: An approach to refining cross-entropy and contrastive learning for optimized learning fusion. In ECAI 2024. IOS Press, 1800–1807

    2024

  13. [21]

    In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

    Multiway-Adapter: Adapting Multimodal Large Language Models for Scalable Image-Text Retrieval. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) . IEEE, 6580–6584

    2024

  14. [22]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. InProceedings ...

    2021

  15. [23]

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. 2021. Zero-shot text-to-image generation. In Proceedings of The International conference on machine learning ICML . PMLR, 8821–8831

    2021

  16. [24]

    Vishvak Murahari, Dhruv Batra, Devi Parikh, and Abhishek Das. 2020. Large- Scale Pretraining for Visual Dialog: A Simple State-of-the-Art Baseline. In Pro- ceedings of The European Conference on Computer Vision ECCV 2020 , Vol. 12363. Springer, 336–352

    2020

  17. [25]

    Denton, Seyed Kamyar Seyed Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, Jonathan Ho, David J

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L. Denton, Seyed Kamyar Seyed Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, Jonathan Ho, David J. Fleet, and Moham- mad Norouzi. 2022. Photorealistic Text-to-Image Diffusion Models...

    2022

  18. [26]

    Kuniaki Saito, Kihyuk Sohn, Xiang Zhang, Chun-Liang Li, Chen-Yu Lee, Kate Saenko, and Tomas Pfister. 2023. Pic2Word: Mapping Pictures to Words for Zero-shot Composed Image Retrieval. In Proceedings of The IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 202...

    2023

  19. [27]

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022. High-Resolution Image Synthesis with Latent Diffusion Mod- els. In Proceedings of The IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022,. IEEE, 10674–10685

    2022

  20. [28]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurélien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lam- ple. 2023. LLaMA: Open and Efficient Foundation ...

    2023 arXiv

  21. [29]

    Yongquan Wan, Wenhai Wang, Guobing Zou, and Bofeng Zhang. 2024. Cross- modal feature alignment and fusion for composed image retrieval. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR

    2024

  22. [30]

    Antonio Tejero-de Pablos. 2024. Complementary-Contradictory Feature Regular- ization Against Multimodal Overfitting. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, W ACV 2024. 5679–5688

    2024

  23. [31]

    Hui Wu, Yupeng Gao, Xiaoxiao Guo, Ziad Al-Halah, Steven Rennie, Kristen Grauman, and Rogério Feris. 2021. Fashion IQ: A New Dataset Towards Retrieving Images by Natural Language Feedback. In Proceedings of The IEEE Conference on Computer Vision and Pattern Recognition, CVPR 20...

    2021

  24. [32]

    Tianyu Wu, Shizhu He, Jingping Liu, Siqi Sun, Kang Liu, Qing-Long Han, and Yang Tang. 2023. A Brief Overview of ChatGPT: The History, Status Quo and Potential Future Development. IEEE/CAA Journal of Automatica Sinica 10, 5 (2023), 1122–1136

    2023

  25. [33]

    Wenhui Wang, Hangbo Bao, Li Dong, Johan Bjorck, Zhiliang Peng, Qiang Liu, Kriti Aggarwal, Owais Khan Mohammed, Saksham Singhal, Subhojit Som, and Furu Wei. 2022. Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Tasks. CoRR abs/2208.10442 (2022)

    2022 arXiv

  26. [34]

    Le Xue, Manli Shu, Anas Awadalla, Jun Wang, An Yan, Senthil Purushwalkam, Honglu Zhou, Viraj Prabhu, Yutong Dai, Michael S. Ryoo, Shrikant Kendre, Jieyu Zhang, Can Qin, Shu Zhang, Chia-Chih Chen, Ning Yu, Juntao Tan, Tu- lika Manoj Awalgaonkar, Shelby Heinecke, Huan Wang, Yeji...

    2024

  27. [35]

    Zixuan Yi, Zijun Long, et al . 2025. Enhancing Recommender Systems: Deep Modality Alignment with Large Multi-Modal Encoders. ACM Transactions on Recommender Systems 3, 4 (2025), 1–25

    2025

  28. [36]

    Mengwei Xu, Wangsong Yin, Dongqi Cai, Rongjie Yi, Daliang Xu, Qipeng Wang, Bingyang Wu, Yihao Zhao, Chen Yang, Shihe Wang, Qiyang Zhang, Zhenyan Lu, Li Zhang, Shangguang Wang, Yuanchun Li, Yunxin Liu, Xin Jin, and Xuanzhe Liu

  29. [37]

    CoRR abs/2401.08092 (2024)

    A Survey of Resource-efficient LLM and Multimodal Foundation Models. CoRR abs/2401.08092 (2024)

    2024 arXiv

  30. [38]

    Yuexiang Zhai, Shengbang Tong, Xiao Li, Mu Cai, Qing Qu, Yong Jae Lee, and Yi Ma. 2024. Investigating the catastrophic forgetting in multimodal large language model fine-tuning. In Proceedings of The Conference on Parsimony and Learning . 202–227

    2024

  31. [39]

    Hongyi Zhu, Jia-Hong Huang, Stevan Rudinac, and Evangelos Kanoulas. 2024. En- hancing Interactive Image Retrieval With Query Rewriting Using Large Language Models and Vision Language Models. In Proceedings of the 2024 International Conference on Multimedia Retrieval, ICMR 2024...

    2024

  32. [40]

    Hee Suk Yoon, Eunseop Yoon, et al. 2024. BI-MDRG: Bridging Image History in Multimodal Dialogue Response Generation. In Proceedings of the The European Conference on Computer Vision ECCV 2024. , Vol. 15089. Springer, 378–396

    2024

  33. [41]

    Yifei Yuan and Wai Lam. 2021. Conversational Fashion Image Retrieval via Multiturn Natural Language Feedback. In Proceedings of The International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR

    2021

  34. [2023]

    IEEE Trans

    Diffusion Models in Vision: A Survey. IEEE Trans. Pattern Anal. Mach. Intell. 45, 9 (2023), 10850–10869

    2023

  35. [2024]

    In Proceedings of the Annual Meet- ing of the Association for Computational Linguistics, ACL 2024

    Generative Cross-Modal Retrieval: Memorizing Images in Multimodal Language Models for Retrieval and Beyond. In Proceedings of the Annual Meet- ing of the Association for Computational Linguistics, ACL 2024 . Association for Computational Linguistics, 11851–11861

    2024

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.