CoCo-IR: Contextual Composed Image Retrieval

Kaifeng Chen; Liang-Yan Gui; Madhuri Shanbhogue; Mojtaba Seyedhosseini; Shengcao Cao; Tanmaya Shekhar Dabral; Yu-Xiong Wang; Zhe Li; Zhongli Ding

REVIEW 3 major objections 4 minor 39 references

CoCo-IR: Contextual Composed Image Retrieval

T0 review · 3 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that multi-turn, contextual composed image retrieval can be solved by a single large multimodal model that re-derives a transformable image embedding from the full dialogue history, and that this native full-context…

desk verdict Genuinely new task and strong single-turn results, but the headline multi-turn metric is teacher-forced and doesn't test recovery from errors; deserves review with a request for free-running evaluation. read the letter →

arxiv 2608.05149 v1 pith:3AOHQ7U2 submitted 2026-08-05 cs.CV

Shengcao Cao , Tanmaya Shekhar Dabral , Zhongli Ding , Madhuri Shanbhogue , Kaifeng Chen , Zhe Li , Mojtaba Seyedhosseini , Yu-Xiong Wang

show 1 more author

Liang-Yan Gui

This is my paper

classification cs.CV

keywords contextualcomposedimageretrievalmulti-turnlargemultimodalmodeltransformableembeddingsyntheticdatagenerationhardnegativeminingcontrastivelearning

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that the single-turn paradigm of composed image retrieval can be extended to multi-turn, contextual retrieval, and that a large multimodal model trained end-to-end is the right way to do it. On a new benchmark of 1–4 turn dialogues, the proposed TIE model achieves 44.1% 4-turn Recall@1, versus 28.2% for the best single-turn baseline even when that baseline is given an oracle summary of the dialogue history. The same model also sets new state-of-the-art single-turn scores on CIRR (38.7% R@1) and CIRCO (39.4% mAP@5) while training on far fewer samples than prior synthetic-data methods. The paper also contributes an autonomous data engine that uses large multimodal models to synthesize instructions, self-reflect on quality, and mine verified hard negatives.

What carries the argument

The model TIE is built on a large multimodal model and adds three ingredients. A dedicated token at the end of each turn acts as a global information bottleneck: the model must aggregate the whole preceding dialogue into a compact embedding, rather than reading off the last text token. A hybrid attention scheme uses bidirectional attention within a turn for deep image-text fusion and causal attention across turns to respect temporal order. The resulting query and target embeddings are trained with an InfoNCE-style contrastive loss, and the training data is generated by an LMM-driven engine that discovers transformation pairs, synthesizes instructions with self-scored quality, rewrites later instructions to reference earlier context, and uses LMM verification to mine hard negatives.

What would settle it

Run the CoCo-IR 4-turn evaluation in self-driven mode: after each turn, feed the model's own top-1 retrieval (rather than the ground-truth image) as the visual context for the next turn, and recompute the fraction of sessions in which all four targets appear in the top-k. If this number falls far below the reported 44.1% R@1, the multi-turn robustness claim is not established.

Watch

Extended reading notes

Core claim

The central claim is that Contextual Composed Image Retrieval (CoCo-IR) is a well-posed task and that it can be solved by a single large multimodal model that consumes the entire interleaved history of images and instructions and outputs a 'transformable' image embedding that is re-derived at every turn. According to the authors, this native full-context modeling is necessary: when single-turn CIR models are adapted by concatenating instructions, using only the latest image, or even receiving an oracle LMM-produced summary of the conversation, their performance collapses with dialogue length, while TIE maintains 44.1% Recall@1 on 4-turn interactions. The paper further claims that the same architecture and data engine generalize to standard single-turn CIR, achieving state-of-the-art results on CIRR and CIRCO, and that the approach is data-efficient, surpassing a strong baseline trained on 36.7M samples with only 1.28M training samples.

Load-bearing premise

The load-bearing premise is that the benchmark's m-turn Recall@k metric, which feeds the model the true history at every turn, measures the model's ability to sustain a real, uninterrupted interactive search; if failures in earlier turns would change the course of the dialogue, the reported numbers may not reflect real-world robustness.

Editorial extensions

If this is right

If correct, interactive visual search no longer requires a user to restart a query from scratch after each refinement.
The results imply that a single large multimodal model can serve as both the dialogue-state reasoner and the embedding model, making separate summarizers or memory modules unnecessary for this task.
The data engine's ability to reach state-of-the-art single-turn CIR with far fewer samples suggests that autonomous, verified synthetic data can substitute for large hand-annotated or weakly supervised corpora.
The performance gap between full-context TIE and oracle-summarized baselines indicates that compressing multimodal history into a text prompt is lossy for retrieval.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A corollary the paper leaves implicit is that the same TIE design could be applied to other embedding tasks where context builds over turns, such as conversational search over documents or video frames, not just images.
The evaluation metric is teacher-forced: it feeds the ground-truth image from turn t−1 when computing turn t. A natural extension is to test in self-driven mode, feeding back the model's own top-1 retrieval, and compare how quickly accuracy degrades.
The claimed advantage over oracle summarization suggests a testable hypothesis: retrieval quality under full-context modeling should degrade gracefully with context length, whereas text-summarized context should degrade sharply once details needed for later instructions are compressed away.
The synthetic data engine's hard-negative mining loop, trained by an earlier model and verified by an LMM, could in principle reinforce biases of the verifying LMM; a human study comparing verification agreement would bound how much that matters.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

First, the good news. The paper introduces a genuine new problem — multi-turn CIR with interleaved images and text at every turn — and it's cleanly distinguished from IRR/MAI, which are fashion-only with text-only later turns. The TIE architecture is a sensible extension of an LMM: the token as bottleneck and within-turn bidirectional attention are well motivated, and the ablations support both choices. The single-turn results on CIRR and CIRCO are state-of-the-art and obtained with far less training data than MagicLens, which is a real outcome. The autonomous data engine has a plausible design (self-reflection, LMM-verified hard negatives), and the 90.5% human validity check on the evaluation set is decent evidence that the benchmark isn't garbage.

Now the soft spots, and the biggest one is the metric. Section 3.1's m-turn Recall@k is defined by feeding ground-truth history at every turn: the model sees I0, T1, I1*, ..., I*_{t-1}, Tt. So a model that fails at turn 1 still gets the correct image for turn 2. The text says this metric 'penalizes off-trajectory compounding errors' and validates 'the entire exploratory journey,' but the math only tests a conjunction of teacher-forced per-turn recalls. It does not test whether the model can sustain a trajectory after its own mistake. That is not a minor quibble; it's the central multi-turn claim. The supplementary final-turn metric also uses ground-truth history and only relaxes the conjunction, so it doesn't fix the problem. The stress-test note is right, and it holds up on reading the paper. The fix is straightforward: add a free-running rollout evaluation (or at least condition on the model's own previous top-1) and report both.

Second, the benchmark is self-generated with the same LMM engine that creates training data. Cluster-level splits reduce leakage, and the human verification helps, but the circularity burden is real. Release the benchmark and code; that would answer most of it.

Third, the baselines are adapted single-turn models, not natively multi-turn embedding models. The paper argues this is the best possible comparison, which is fair, but adding one natively multi-turn baseline (e.g., a variant of TIE that uses last-token embedding, or a trained IRR-style model) would strengthen the claim that the architecture matters rather than the data.

Fourth, no error bars on the headline numbers, and a handful of evaluation queries at 4 turns (392) means the 44.1 vs 28.2 gap has wide uncertainty. Minor, but worth reporting.

Who is this for? Researchers working on interactive retrieval, LMM-based embeddings, and synthetic data pipelines. They'll find the task formulation and data engine useful even if the metric needs work.

My recommendation: send to peer review, but with a clear request to address the teacher-forced metric, either by correcting the description or adding a free-running evaluation. This is a serious contribution with a fixable flaw, not a reject.

Referee Report

3 major / 4 minor

Summary. The paper introduces Contextual Composed Image Retrieval (CoCo-IR), a multi-turn variant of composed image retrieval in which a user progressively refines a search through interleaved images and instructions. The authors propose TIE, an LMM-based model with a special <EMB> embedding token and hybrid attention, trained contrastively on a large autonomously generated dataset. They report state-of-the-art single-turn results on CIRR and CIRCO, and strong multi-turn results on a new CoCo-IR benchmark, where TIE-12B reaches 44.1% 4-turn Recall@1 versus 28.2% for the best adapted baseline. The paper also contributes a data engine with self-reflection-based instruction filtering and LMM-verified hard-negative mining.

Significance. If the multi-turn results hold, this would be a meaningful step toward interactive visual search: TIE natively consumes full interaction histories, and the single-turn results on external human-annotated benchmarks (CIRR R@1 38.7, CIRCO mAP@5 39.4 with roughly 14x fewer training samples than MagicLens) are strong and credible. The ablations support the individual design choices (bidirectional within-turn attention, single-side contrastive loss, and the <EMB> token), and the supplementary material provides reproducible training details and a careful comparison of baseline adaptation strategies. However, the central multi-turn claim rests on a teacher-forced evaluation metric and a self-generated benchmark, both of which need to be addressed before the headline result can be interpreted as evidence of robust multi-turn retrieval behavior.

major comments (3)

[Section 3.1, m-turn Recall@k definition] The m-turn Recall@k definition conditions every turn on the ground-truth history (I0, T1, I1*, ..., I*_{t-1}, Tt). This makes the metric a conjunction of teacher-forced per-turn recalls rather than a trajectory-level measure: a miss at turn 1 does not alter the input at turn 2, so off-trajectory errors never compound and the model is never tested on recovery or on interpreting instructions after a diverged context. The same paragraph claims the metric 'penalizes off-trajectory compounding errors' and validates 'the entire exploratory journey,' which the formal definition cannot do. Consequently, the headline 44.1% 4-turn R@1 in Table 2 establishes strong per-turn contextual understanding when given oracle history, but not robust completion of an unbroken multi-turn session. Please add a free-running evaluation (feeding the model's own retrieved image at each turn) or, at minimum, explicitly reframe the claims to per-turn contextual understanding under teacher forcing.
[Section 3.3 and Supplementary B] The CoCo-IR evaluation benchmark is generated by the same LMM pipeline that creates the training data: Gemini 2.5 Flash synthesizes instructions, performs self-reflection scoring, rewrites multi-turn instructions, and verifies hard negatives, and the evaluation split is filtered by those same scores. This creates a distributional overlap between training and evaluation that can inflate reported multi-turn gains, particularly because the model and the benchmark share the same instruction-generation distribution. The human verification reported in Supplementary B (on 0.14%–7.65% of the evaluation samples) checks trajectory validity, not independence from the training distribution. Please report results on an independently authored multi-turn benchmark, or at least on a human-rewritten subset of the evaluation split, and quantify the overlap between training and evaluation instructions.
[Table 2 and Supplementary B (evaluation split sizes)] The 4-turn evaluation split contains only 392 queries and the 3-turn split 883 queries, with no confidence intervals or significance tests reported. Since the columns are disjoint splits, the observed 44.1% versus 28.2% gap for 4-turn R@1 could be sensitive to a small number of trajectories. Please report bootstrap confidence intervals or significance tests, and consider validating on a larger held-out set of long interactions to support the claim of 'robust performance' on 4-turn dialogues.

minor comments (4)

[Section 3.1, notation] The formula 'Vm t=1(I ∗ t ∈ Rk t )' appears to be a rendering error; the intended meaning is a conjunction over t, e.g., ∧_{t=1}^m (I*_t ∈ R^k_t). Please fix the notation.
[Table 4] The ablation rows in Table 4 are difficult to read because the checkmark columns do not clearly indicate which components are enabled or ablated in each row. Please restructure the table with explicit row labels such as 'w/o BA', 'w/o SC', and 'w/o ET'.
[Introduction and Table 5] The introduction states that TIE achieves state-of-the-art performance with '14× fewer training samples than prior work [37]', but Table 5 reports that TIE-4B with 320K samples already surpasses MagicLens, which corresponds to more than 100× fewer samples. Please clarify which operating point the 14× claim refers to.
[Figures and supplementary text] Several passages contain missing spaces or typos, e.g., 'andournovel' and 'CoCo-IRtask' in Figure 1, and 'Overvall' in Supplementary C. A thorough proofreading pass is needed.

Circularity Check

2 steps flagged · score 5.0 of 10

The 4-turn R@1 is a teacher-forced conjunction of per-turn recalls on a Gemini-generated benchmark, so the multi-turn advantage is partly self-referential; external single-turn results keep some independence.

other [Section 3.3, Data Engine steps 2-5; Section 4.3, CoCo-IR evaluation]
"For the training set, we use a more relaxed threshold to maintain data diversity, while for the evaluation set, we enforce the highest possible scores to ensure benchmark quality. ... We then prompt Gemini 2.5 Flash to rewrite instructions (starting from the second turn) based on previous context."

The CoCo-IR evaluation trajectories are generated and filtered by the same Gemini-based data engine used to produce the training data: Gemini synthesizes the transformation instructions, assigns self-reflection quality scores that determine the evaluation split, rewrites multi-turn instructions, and verifies hard negatives. There is no externally authored ground-truth trajectory for the multi-turn benchmark. Consequently the headline 4-turn 44.1% R@1 is a score on the authors' own synthetic distribution, which is also the training distribution; the 'prediction' of multi-turn superiority is a self-consistent fit to that distribution rather than a test against independent ground truth.
self definitional [Section 3.1, Evaluation Metric]
"let R^k_t be the set of top-k images retrieved by the model at turn t, given the ground-truth history up to step t: (I0, T1, I*_1, ..., I*_{t-1}, Tt). The entire retrieval is marked as correct if and only if V^m_{t=1}(I*_t in R^k_t). ... it penalizes off-trajectory compounding errors and ensures the model is rewarded only when it successfully supports the user's entire exploratory journey."

By construction the metric feeds the ground-truth target I*_{t-1} at every turn, so a failure at turn t-1 does not alter the model's input at turn t; the trajectory can never diverge from the benchmark path. The m-turn score is therefore exactly the conjunction of m teacher-forced per-turn recalls. The paper's interpretation that this metric 'penalizes off-trajectory compounding errors' and validates the 'entire exploratory journey' is not derived from the definition; the stated metric cannot detect compounding errors because it never rolls out the model's own retrieved images. Thus the central multi-turn capability claim is supported only by a metric that reduces to independent per-turn successes.

full rationale

The paper's core derivation chain is the TIE model trained on synthetic data and evaluated on its own CoCo-IR benchmark plus external single-turn benchmarks. The single-turn results on FIQ, CIRR, and CIRCO are independent, human-annotated benchmarks and are not circular; they show the retrieval backbone generalizes. The multi-turn evaluation, however, has two self-referential elements. First, the m-turn Recall@k metric is defined with ground-truth history at every turn, so it is a conjunction of teacher-forced per-turn recalls; the paper's claim that it 'penalizes off-trajectory compounding errors' is not supported by the definition. Second, the benchmark trajectories are generated and filtered by the same Gemini-based engine that produced the training data, with human verification covering only a small fraction of samples. These elements make the headline multi-turn advantage (44.1 vs 28.2) partly self-referential. No fitted parameter is renamed as prediction, and no load-bearing self-citation chain is present, so the circularity is partial rather than total.

Assumptions & free parameters 6 free parameters · 6 assumptions · 2 invented entities

The central claims rest on many hand-chosen thresholds and on trusting the same LMM family both to generate training signal and to judge evaluation quality. External single-turn benchmarks provide a partial check, but the multi-turn advantage is measured only on the authors' own artifact.

free parameters (6)

Feature similarity thresholds for pair discovery = [0.7, 0.97] (feature) and [0.5, 0.97] (metadata)
Chosen thresholds determine which image pairs are considered related and drastically shape the generated training distribution.
Self-reflection quality thresholds = training: scores >= 3; evaluation: scores == 5
Filters all training and evaluation data; the evaluation benchmark is deliberately restricted to the highest-scoring synthetic trajectories, affecting metric comparability.
Number of nearest-neighbor candidates for hard negatives = top-10 per source and target; 1 hard negative per training sample
Selects the difficulty of negatives and affects contrastive learning and benchmark difficulty.
Number of training steps and batch size = 10,000 steps, batch 256
Computational budget choices; the paper reports 2.56M total samples seen and uses this for the 14x fewer samples claim.
Temperature tau = 0.1
Contrastive loss temperature fixed by hand; influences embedding sharpness.
Maximum multi-turn chain length = 4 turns
Defines the benchmark's turn distribution and limits the claim to interactions up to 4 turns.

assumptions (6)

domain assumption LMM self-reflection scores are reliable indicators of instruction and image-pair quality.
The data engine filters pairs and instructions based on Gemini's self-assigned 1-5 quality scores (Section 3.3 and Supp B); if these scores are biased, the benchmark and training set inherit the bias.
domain assumption Web image co-occurrence and CLIP feature similarity identify meaningful transformation pairs.
Both discovery strategies in Section 3.3 assume that images that co-occur on a page or fall in neighboring feature clusters are related by a describable transformation.
domain assumption Gemini 2.5 Flash's pairwise judgments reliably distinguish true negatives from false negatives.
Hard negative mining for both training and evaluation relies on an LMM verifier to decide whether a candidate image is worse than the target (Section 3.3, Fig. 4).
domain assumption Ground-truth history at each turn is a valid input for evaluating multi-turn retrieval.
The m-turn Recall definition in Section 3.1 feeds the ground-truth previous image at every turn; this assumes teacher forcing captures interactive behavior, which is contradicted by the paper's prose about unbroken chains.
domain assumption The image cluster-level split prevents train/eval leakage.
Section 3.3 states evaluation clusters are held out, but the split is based on the same CLIP features used for discovery, and leakage via similar instructions or shared images is not fully analyzed.
domain assumption Gemma 3 pretrained weights and SigLIP-SO400M provide a sufficient base for retrieval.
The model is initialized from public pretrained checkpoints; this is background tooling, not a novel postulate.

invented entities (2)

CoCo-IR benchmark (21,084 single-turn, 3,129 two-turn, 883 three-turn, 392 four-turn queries)
purpose: Primary evaluation set for the new multi-turn retrieval task.
The benchmark is generated by the authors' own LMM pipeline and is not released in the preprint, so external researchers cannot verify it independently.
Transformable Image Embedding (TIE) model with special <EMB> token and hybrid attention
purpose: Produces evolving query embeddings that summarize the full multi-turn history.
TIE's benefits are demonstrated only on benchmarks built by the same pipeline; no independent implementation or release is provided, though the strong CIRCO/CIRR scores offer indirect external evidence for the base retriever.

how reviews work

0 comments

Cite this review

Pith. "Pith review of CoCo-IR: Contextual Composed Image Retrieval." pith.science (2026). https://pith.science/paper/3AOHQ7U2

@misc{pith2026260805149,
  author       = {Pith},
  title        = {Pith review of: CoCo-IR: Contextual Composed Image Retrieval},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3AOHQ7U2}},
  note         = {Machine review of arXiv:2608.05149}
}

read the original abstract

Current instruction-based image retrieval systems are powerful but limited to single-turn interactions, failing to capture the iterative nature of complex, real-world visual searches. To overcome this limitation, we introduce Contextual Composed Image Retrieval (CoCo-IR), a novel task that enables users to progressively refine search results through interactions. We address this new task by proposing a new model based on a Large Multimodal Model (LMM) that functions as a context-aware reasoner for CoCo-IR. Our model interprets the entire interaction history to generate Transformable Image Embeddings (TIE) that evolve across turns. To fuel the model training without expensive human annotations, we develop a fully autonomous, scalable data engine that leverages LMMs to generate high-quality contextual retrieval data, and uses model-guided verification to mine challenging hard negatives. Extensive experiments demonstrate that our approach establishes new state-of-the-art performance: We achieve 39.4 mAP@5 on the challenging single-turn benchmark CIRCO; furthermore, on our new CoCo-IR benchmark, our model maintains robust performance with 44.1 R@1 on 4-turn dialogues, dramatically outperforming existing methods (28.2 4-turn R@1) that fail to handle multi-turn context. Project page: https://CoCo-IR.github.io.

Figures

Figures reproduced from arXiv: 2608.05149 by the authors.

**Figure 1.** Comparison between conventional single-turn Composed Image Retrieval (CIR) and our novel Contextual Composed Image Retrieval (CoCo-IR) task, achieved via our proposed TIE framework. (Top Left) A standard single-turn CIR model successfully processes a simple, isolated instruction. (Top Right) Traditional CIR models are architecturally unable to process multi-turn context with interleaving images and text. Even with a… view at source ↗

**Figure 2.** Overview of our Transformable Image Embedding ( [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

**Figure 3.** Self-reflection during instruction synthesis. [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: Verification of hard negative samples. Hard negatives are images that are visually and semantically similar to the ground-truth target (middle) but fail to perfectly satisfy the instruction (left). Here, the candidate negative (right) correctly matches the “blurred li…

**Figure 6.** Figure 6: Comparison of context encoding strategies for [PITH_FULL_IMAGE:figures/full_fig_p022_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

39 extracted references · 23 canonical work pages

[1]

In: NeurIPS (2022)

Alayrac, J.B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., Ring, R., Rutherford, E., Cabi, S., Han, T., Gong, Z., Samangooei, S., Monteiro, M., Menick, J., Borgeaud, S., Brock, A., Nematzadeh, A., Sharifzadeh, S., Binkowski, M., Barreira, R., Vinyals, O., Zisserman, A., Simonyan, K.: Flamingo: A...

work page 2022
[2]

In: ICCV (2023)

Baldrati, A., Agnolucci, L., Bertini, M., Del Bimbo, A.: Zero-shot composed image retrieval with textual inversion. In: ICCV (2023)

2023
[3]

In: CVPR (2023)

Brooks, T., Holynski, A., Efros, A.A.: InstructPix2Pix: Learning to follow image editing instructions. In: CVPR (2023)

work page 2023
[4]

In: ICLR (2023)

Chen, X., Wang, X., Changpinyo, S., Piergiovanni, A., Padlewski, P., Salz, D., Goodman, S., Grycner, A., Mustafa, B., Beyer, L., Kolesnikov, A., Puigcerver, J., Ding, N., Rong, K., Akbari, H., Mishra, G., Xue, L., Thapliyal, A.V., Bradbury, J., Kuo, W., Seyedhosseini, M., Jia, C., Ayan, B.K., Ruiz, C.R., Steiner, A.P., Angelova, A., Zhai, X., Houlsby, N.,...

work page 2023
[5]

Chen, Y., Hu, H., Luan, Y., Sun, H., Changpinyo, S., Ritter, A., Chang, M.: Can pre-trained vision and language models answer visual information-seeking ques- tions? In: EMNLP (2023)

work page 2023
[6]

In: ICLR (2025)

Chen, Y., Yang, Z., Xu, J., Peng, Y.: MAI: A multi-turn aggregation-iteration model for composed image retrieval. In: ICLR (2025)

work page 2025
[7]

TMLR (2024)

Gu, G., Chun, S., Kim, W., Jun, H., Kang, Y., Yun, S.: CompoDiff: Versatile composed image retrieval with latent diffusion. TMLR (2024)

work page 2024
[8]

In: CVPR (2024)

Gu, G., Chun, S., Kim, W., Kang, Y., Yun, S.: Language-only training of zero-shot composed image retrieval. In: CVPR (2024)

2024

Show all 39 references

[9]

In: CVPR (2016)

He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR (2016)

2016
[10]

arXiv preprint arXiv:2407.12580 (2024)

Jiang, T., Song, M., Zhang, Z., Huang, H., Deng, W., Sun, F., Zhang, Q., Wang, D., Zhuang, F.: E5-V: Universal embeddings with multimodal large language models. arXiv preprint arXiv:2407.12580 (2024)

2024 arXiv
[11]

Jordan, K., Jin, Y., Boza, V., Jiacheng, Y., Cesista, F., Newhouse, L., Bernstein, J.: Muon: An optimizer for hidden layers in neural networks (2024),https:// kellerjordan.github.io/posts/muon/

2024
[12]

In: ICLR (2024)

Karthik, S., Roth, K., Mancini, M., Akata, Z.: Vision-by-language for training-free compositional image retrieval. In: ICLR (2024)

2024
[13]

Li, B., Zhang, K., Zhang, H., Guo, D., Zhang, R., Li, F., Zhang, Y., Liu, Z., Li, C.: LLaVA-NeXT: Stronger LLMs supercharge multimodal capabilities in the wild (May 2024),https://llava-vl.github.io/blog/2024-05-10-llava-next- stronger-llms/

2024
[14]

In: ICML (2023)

Li, J., Li, D., Savarese, S., Hoi, S.: BLIP-2: Bootstrapping language-image pre- training with frozen image encoders and large language models. In: ICML (2023)

2023
[15]

In: ICML (2022)

Li, J., Li, D., Xiong, C., Hoi, S.: BLIP: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In: ICML (2022)

2022
[16]

In: NeurIPS (2021)

Li, J., Selvaraju, R., Gotmare, A., Joty, S., Xiong, C., Hoi, S.C.H.: Align before fuse: Vision and language representation learning with momentum distillation. In: NeurIPS (2021)

2021
[17]

In: CVPR (2024) CoCo-IR: Contextual Composed Image Retrieval 17

Liu, H., Li, C., Li, Y., Lee, Y.J.: Improved baselines with visual instruction tuning. In: CVPR (2024) CoCo-IR: Contextual Composed Image Retrieval 17

2024
[18]

Liu, H., Li, C., Li, Y., Li, B., Zhang, Y., Shen, S., Lee, Y.J.: LLaVA-NeXT: Im- proved reasoning, OCR, and world knowledge (January 2024),https://llava- vl.github.io/blog/2024-01-30-llava-next/

2024
[19]

In: NeurIPS (2023)

Liu, H., Li, C., Wu, Q., Lee, Y.J.: Visual instruction tuning. In: NeurIPS (2023)

2023
[20]

arXiv preprint arXiv:2502.16982 (2025)

Liu, J., Su, J., Yao, X., Jiang, Z., Lai, G., Du, Y., Qin, Y., Xu, W., Lu, E., Yan, J., et al.: Muon is scalable for LLM training. arXiv preprint arXiv:2502.16982 (2025)

2025 arXiv
[21]

In: ICCV (2021)

Liu, Z., Rodriguez-Opazo, C., Teney, D., Gould, S.: Image retrieval on real-life images with pre-trained vision-and-language models. In: ICCV (2021)

2021
[22]

In: ICLR (2025)

Maninis, K.K., Chen, K., Ghosh, S., Karpur, A., Chen, K., Xia, Y., Cao, B., Salz, D., Han, G., Dlabal, J., Gnanapragasam, D., Seyedhosseini, M., Zhou, H., Araujo, A.: TIPS: Text-image pretraining with spatial awareness. In: ICLR (2025)

2025
[23]

arXiv preprint arXiv:1807.03748 (2018)

Oord, A.v.d., Li, Y., Vinyals, O.: Representation learning with contrastive predic- tive coding. arXiv preprint arXiv:1807.03748 (2018)

2018 arXiv
[24]

In: ICML (2021)

Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., Sutskever, I.: Learning transferable visual models from natural language supervision. In: ICML (2021)

2021
[25]

In: CVPR (2023)

Saito, K., Sohn, K., Zhang, X., Li, C.L., Lee, C.Y., Saenko, K., Pfister, T.: Pic2Word: Mapping pictures to words for zero-shot composed image retrieval. In: CVPR (2023)

2023
[26]

In: NeurIPS (2022)

Schuhmann, C., Beaumont, R., Vencu, R., Gordon, C.W., Wightman, R., Cherti, M., Coombes, T., Katta, A., Mullis, C., Wortsman, M., Schramowski, P., Kun- durthy, S.R., Crowson, K., Schmidt, L., Kaczmarczyk, R., Jitsev, J.: LAION-5B: An open large-scale dataset for training next ...

2022
[27]

arXiv preprint arXiv:2507.06261 (2025)

Team, G.: Gemini 2.5: Pushing the frontier with advanced reasoning, multi- modality, long context, and next generation agentic capabilities. arXiv preprint arXiv:2507.06261 (2025)

2025 arXiv
[28]

arXiv preprint arXiv:2503.19786 (2025)

Team, G.: Gemma 3 technical report. arXiv preprint arXiv:2503.19786 (2025)

2025 arXiv
[29]

arXiv preprint arXiv:2502.14786 (2025)

Tschannen, M., Gritsenko, A., Wang, X., Naeem, M.F., Alabdulmohsin, I., Parthasarathy, N., Evans, T., Beyer, L., Xia, Y., Mustafa, B., Hénaff, O., Harm- sen, J., Steiner, A., Zhai, X.: SigLIP 2: Multilingual vision-language encoders with improved semantic understanding, locali...

2025 arXiv
[30]

In: CVPR (2019)

Vo, N., Jiang, L., Sun, C., Murphy, K., Li, L.J., Fei-Fei, L., Hays, J.: Composing text and image for image retrieval - an empirical odyssey. In: CVPR (2019)

2019
[31]

In: ACM MM (2023)

Wei, H., Wang, S., Xue, Z., Chen, S., Huang, Q.: Conversational composed retrieval with iterative sequence refinement. In: ACM MM (2023)

2023
[32]

In: CVPR (2021)

Wu, H.,Gao, Y., Guo,X., Al-Halah, Z., Rennie,S., Grauman, K., Feris, R.:Fashion IQ: A new dataset towards retrieving images by natural language feedback. In: CVPR (2021)

2021
[33]

In: SIGIR (2024)

Yang, Z., Xue, D., Qian, S., Dong, W., Xu, C.: LDRE: LLM-based divergent rea- soning and ensemble for zero-shot composed image retrieval. In: SIGIR (2024)

2024
[34]

TMLR (2022)

Yu, J., Wang, Z., Vasudevan, V., Yeung, L., Seyedhosseini, M., Wu, Y.: CoCa: Contrastive captioners are image-text foundation models. TMLR (2022)

2022
[35]

In: CVPR (2022)

Zhai, X., Kolesnikov, A., Houlsby, N., Beyer, L.: Scaling vision transformers. In: CVPR (2022)

2022
[36]

In: ICCV (2023)

Zhai, X., Mustafa, B., Kolesnikov, A., Beyer, L.: Sigmoid loss for language image pre-training. In: ICCV (2023)

2023
[37]

In: ICML (2024) 18 S

Zhang, K., Luan, Y., Hu, H., Lee, K., Qiao, S., Chen, W., Su, Y., Chang, M.W.: MagicLens: Self-supervised image retrieval with open-ended instructions. In: ICML (2024) 18 S. Cao et al

2024
[38]

arXiv preprint arXiv:2506.05176 (2025)

Zhang, Y., Li, M., Long, D., Zhang, X., Lin, H., Yang, B., Xie, P., Yang, A., Liu, D., Lin, J., Huang, F., Zhou, J.: Qwen3 Embedding: Advancing text embedding and reranking through foundation models. arXiv preprint arXiv:2506.05176 (2025)

2025 arXiv
[39]

floral design,

Zhou, J., Xiong, Y., Liu, Z., Liu, Z., Xiao, S., Wang, Y., Zhao, B., Zhang, C.J., Lian, D.: MegaPairs: Massive data synthesis for universal multimodal retrieval. In: ACL (2025) CoCo-IR: Contextual Composed Image Retrieval 1 This supplementary material provides additional detai...

2025

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

In: NeurIPS (2022)

Alayrac, J.B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., Ring, R., Rutherford, E., Cabi, S., Han, T., Gong, Z., Samangooei, S., Monteiro, M., Menick, J., Borgeaud, S., Brock, A., Nematzadeh, A., Sharifzadeh, S., Binkowski, M., Barreira, R., Vinyals, O., Zisserman, A., Simonyan, K.: Flamingo: A...

work page 2022

[2] [2]

In: ICCV (2023)

Baldrati, A., Agnolucci, L., Bertini, M., Del Bimbo, A.: Zero-shot composed image retrieval with textual inversion. In: ICCV (2023)

2023

[3] [3]

In: CVPR (2023)

Brooks, T., Holynski, A., Efros, A.A.: InstructPix2Pix: Learning to follow image editing instructions. In: CVPR (2023)

work page 2023

[4] [4]

In: ICLR (2023)

Chen, X., Wang, X., Changpinyo, S., Piergiovanni, A., Padlewski, P., Salz, D., Goodman, S., Grycner, A., Mustafa, B., Beyer, L., Kolesnikov, A., Puigcerver, J., Ding, N., Rong, K., Akbari, H., Mishra, G., Xue, L., Thapliyal, A.V., Bradbury, J., Kuo, W., Seyedhosseini, M., Jia, C., Ayan, B.K., Ruiz, C.R., Steiner, A.P., Angelova, A., Zhai, X., Houlsby, N.,...

work page 2023

[5] [5]

Chen, Y., Hu, H., Luan, Y., Sun, H., Changpinyo, S., Ritter, A., Chang, M.: Can pre-trained vision and language models answer visual information-seeking ques- tions? In: EMNLP (2023)

work page 2023

[6] [6]

In: ICLR (2025)

Chen, Y., Yang, Z., Xu, J., Peng, Y.: MAI: A multi-turn aggregation-iteration model for composed image retrieval. In: ICLR (2025)

work page 2025

[7] [7]

TMLR (2024)

Gu, G., Chun, S., Kim, W., Jun, H., Kang, Y., Yun, S.: CompoDiff: Versatile composed image retrieval with latent diffusion. TMLR (2024)

work page 2024

[8] [8]

In: CVPR (2024)

Gu, G., Chun, S., Kim, W., Kang, Y., Yun, S.: Language-only training of zero-shot composed image retrieval. In: CVPR (2024)

2024