Pith. sign in

REVIEW 4 major objections 5 minor 37 references

QuRe: Query-Relevant Retrieval through Hard Negative Sampling in Composed Image Retrieval

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read QuRe is a training method for composed image retrieval that replaces all-batch negatives with one hard negative sampled from between the two sharpest relevance-score drops after the target, and it reports state-of-the-art results on…

desk verdict Solid CIR paper with a clever hard-negative sampler, but the human-preference win is a 0.1 pp gap and the boundary assumption needs direct validation. read the letter →

arxiv 2507.12416 v1 pith:UNHQLDO5 submitted 2025-07-16 cs.CV cs.AI

Jaehyun Kwak , Ramahdani Muhammad Izaaz Inhar , Se-Young Yun , Sung-Ju Lee This is my paper
classification cs.CVcs.AI
keywords composedimageretrievalhardnegativesamplingfalsenegativespaired-preferencetraininghumanpreferenceevaluationFashionIQCIRRzero-shot
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes QuRe, a training method for composed image retrieval (CIR)—retrieving images from a reference image plus a text modification—that aims to rank not just the single annotated target image but every image a user would consider relevant. Existing CIR training treats every non-target image in a batch as a negative, so visually similar relevant images become false negatives and the model learns to suppress them, hurting the quality of the retrieved set. QuRe instead uses a paired-preference objective that compares the target against one sampled hard negative, and selects that hard negative as an image lying between two sharp drops in relevance scores after the target. On FashionIQ and CIRR, QuRe reports state-of-the-art recall, and on a new human-preference dataset it shows the best agreement with user choices, supporting the argument that relevance beyond the target is trainable and measurable.

What carries the argument

The load-bearing mechanism is a query-specific hard-negative set defined from sorted relevance scores. For each query, images are ranked by $s(x_I, x_T, I) = Q(E_{\text{img}}(x_I), x_T) \cdot Q(E_{\text{img}}(I))/\tau$, where $E_{\text{img}}$ is the image encoder, $Q$ the Q-Former fusion module, and $\tau$ the learned temperature. After locating the target in the sorted list, the method takes the two largest positive score differences below it and collects images between those drops, excluding the target; one image is sampled uniformly from this set per epoch. The training loss is the negative log-likelihood of a paired-preference model, $\mathcal{L} = -\log \sigma(s(x_I,x_T,I_p) - s(x_I,x_T,I_n))$, which steers the model to keep the target above the hard negative while leaving other relevant images free to rank above the target.

What would settle it

One could take a random sample of queries from FashionIQ and CIRR, have human annotators label every image in the band between the two largest score drops as relevant or irrelevant to the query, and check whether the false-negative rate in that band is near zero and whether the band excludes relevant images above it. Alternatively, one could artificially smooth or perturb the relevance-score curves and measure whether QuRe's improvement over the all-corpus baseline disappears; if the two-drop boundary does not correspond to semantic shifts, smoothing should leave performance essentially unchanged.

Watch

Extended reading notes

Core claim

QuRe's central claim is that a CIR model can be trained to retrieve both the target image and other relevant images by replacing batch contrastive loss with a pairwise reward-model objective over carefully chosen hard negatives. Using the model's own relevance scores, the corpus is sorted per query; the hard negative set is the band of images between the two largest score drops that occur after the target, a region that the paper argues separates false negatives from true hard negatives. Training then maximizes the probability that the target scores above a uniformly sampled image from that band. The paper reports state-of-the-art results on FashionIQ and CIRR, and the best preference rate on the newly collected HP-FashionIQ benchmark.

Load-bearing premise

The method assumes that after a short warm-up, the model's own relevance scores are reliable enough that the two steepest downward steps after the target mark the true semantic boundary, with false negatives above and true hard negatives below; if those score landscapes are noisy or attribute changes do not produce clear drops, the selected hard negatives are arbitrary and the reported gains would not follow.

Editorial extensions

If this is right

  • If QuRe is right, retrieval quality in CIR should be evaluated not only by whether the exact target appears in the top-$k$, but by how many relevant images appear; QuRe's score alignment on HP-FashionIQ suggests Recall@k understates user satisfaction.
  • The query-specific hard-negative band makes the effective negative set shrink during training—the paper reports a consistent decrease in set size—so later epochs train on progressively harder examples, a curriculum-like effect that may accelerate convergence.
  • Because the hard-negative definition uses only relevance scores and the target's rank, it transfers to datasets with a single positive annotation per query without needing class labels; zero-shot results on CIRCO support this transfer.
  • Models trained with QuRe should generalize better at small $k$: the paper reports its largest gains on Recall@10 over the strongest baseline, which is exactly the regime where false negatives most hurt user-visible results.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • I would expect the same two-drop sampling idea to transfer to other single-positive retrieval tasks such as product search or video retrieval, because the procedure never relies on class labels; a natural test is applying QuRe to datasets with known multiple ground truths, where false-negative contamination can be measured directly.
  • The HP-FashionIQ preference data opens an alternative evaluation protocol for CIR: instead of ranking models by Recall@k, one can compute preference rates over retrieved sets, which may change model selection in applications where set-level relevance matters more than exact-match recall.
  • A likely refinement of the heuristic is to learn the boundary itself—for instance, training a small classifier to predict where the false-negative/hard-negative split occurs—since the paper's ablation shows that the fixed top-$k$ after target degrades over time while the two-drop rule stays stable, suggesting the rule is capturing a real but noisy signal.
  • The described warm-up phase matters: selecting hard negatives from an untrained model would include many easy examples, so QuRe implicitly depends on the initial contrastive training being good enough to produce meaningful score cliffs; this dependency is worth isolating in future ablations.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes QuRe, a composed image retrieval (CIR) method that replaces the standard contrastive objective with a pairwise reward-model objective (Bradley-Terry) and introduces a hard-negative sampling strategy. After a warm-up phase, the image corpus is ranked by the model's current relevance scores, and a hard-negative set is defined as the images between the two largest relevance-score drops below the target score. The model is trained to rank the target above a uniformly sampled hard negative. The authors also introduce HP-FashionIQ, a human-preference dataset built from FashionIQ validation queries in which annotators choose between two retrieval sets. Experiments report state-of-the-art results on FashionIQ and CIRR, the strongest alignment with human preferences on HP-FashionIQ, and improved zero-shot mAP on CIRCO.

Significance. If the central claims hold, QuRe would be a useful contribution: it shifts CIR training from target-only retrieval to a reward-model formulation that can rank relevant non-target images highly, and the HP-FashionIQ dataset provides a new evaluation axis beyond Recall@k. The paper clearly specifies the algorithm and includes ablations that support the value of query-adaptive hard-negative selection over fixed top-k heuristics. The code release and the new human-preference dataset are assets. However, the load-bearing assumption on the hard-negative boundary is not validated with ground truth, the headline SOTA numbers are reported without statistical uncertainty and mix backbones, and the HP-FashionIQ preference-rate margin over CLIP4CIR is 0.10 percentage points, far below any plausible significance threshold. These issues currently prevent accepting the strong claims as stated.

major comments (4)
  1. [Sec. 3.2, Eq. (6)-(7)] The hard negative set is defined as the images between the two largest relevance-score drops after the target. The paper asserts that these drops mark semantic boundaries, citing Xia et al. (2024), but this property is never demonstrated for CIR relevance-score landscapes. Figure 6 shows only two hand-picked queries; Figure 8 shows aggregated score curves but does not label whether the selected range actually excludes false negatives. A quantitative validation is needed; for example, using CIRCO, which provides multiple ground-truth relevant images per query, report the fraction of the selected hard-negative set that is actually relevant (false-negative contamination), and compare the proposed boundary rule against alternatives such as a fixed gap threshold or a percentile-based rule. Without such evidence, the claimed mechanism, namely that the method excludes false negatives and thereby retrieves other relevant images, remains unsupported; the observed training gains could arise from the pairwise objective alone or from selecting near-target negatives regardless of the specific boundary rule.
  2. [Sec. 5.3, Table 4] The claim of the strongest alignment with human preferences rests on a preference-rate difference of 74.55% (QuRe) versus 74.45% (CLIP4CIR), a 0.10 percentage point gap. Given roughly 2,715 annotated queries and the fact that each model appears in only a subset of comparisons, the standard error of these proportions is about one percentage point; the gap is therefore not statistically significant. No confidence intervals or significance tests are reported anywhere in the paper, including for the FashionIQ and CIRR results. Please report bootstrap confidence intervals or query-level significance tests for the headline metrics and temper the 'state-of-the-art' and 'strongest alignment' claims accordingly if the differences are not significant.
  3. [Sec. 5.2, Tables 2 and 3] The main comparisons mix model backbones: QuRe uses BLIP-2 while several baselines (e.g., Bi-BLIP4CIR, CoVR-BLIP, and the reported SPRC numbers) use weaker backbones. The appendix provides controlled-backbone comparisons for Bi-BLIP4CIR and CoVR-2, but the strongest baseline SPRC is not included in those controlled experiments. From Appendix Table 7, QuRe with the BLIP backbone reaches an average FashionIQ recall of 57.08, which is below the SPRC value of 62.13 reported in Table 2. This suggests that a large part of the claimed SOTA margin may be attributable to the BLIP-2 backbone rather than to the proposed method. Please provide a same-backbone comparison with SPRC (and, if feasible, CLIP4CIR) or explicitly state that the main-table numbers are not directly comparable across backbones.
  4. [Sec. 5.4, Figure 4] The ablation compares four hard-negative-set definitions while keeping the same pairwise reward objective, but it does not isolate the contribution of the reward objective itself. A variant that uses the standard contrastive loss with the proposed hard-negative sampling, or a variant using the reward objective with random corpus negatives, is needed to attribute the observed gains. As it stands, the paper does not demonstrate that the reward-model objective contributes beyond what the negative sampling alone would achieve, even though the Abstract and Introduction credit both components.
minor comments (5)
  1. [Sec. 5.2] The text says 'Table 7 presents the evaluation of CIR models on the FashionIQ dataset,' but the main FashionIQ comparison is in Table 2; Table 7 is in the appendix and uses controlled backbones.
  2. [Algorithm 1, line 6] The inequality in the definition of H is written as 's(xI, xT, yI) < s(xI, xT, I)', which is the reverse of the condition in Eq. (7). It should be 's(xI, xT, I) < s(xI, xT, yI)'.
  3. [Sec. 3.1] The statement that the NLL loss 'is equivalent to minimizing the KL divergence between p* and a target distribution p = [1, 0]' is imprecise; the degenerate target distribution would make the KL divergence infinite. Minimizing the NLL is equivalent to minimizing the cross-entropy, whose entropy term is constant with respect to the parameters.
  4. [Sec. 4] The sanity check discards 28 queries where the user's preferred set was not the set with the higher relevance rating. This procedure removes potential preference signals that may be valid (e.g., aesthetic preference over raw relevance) and should be justified or reported with and without the exclusion.
  5. [Sec. 3.2, Eq. (7)] If the two largest drops occur at adjacent positions, the interval [min(k1,k2)+1, max(k1,k2)] can be empty. The paper does not specify a fallback in this case, which should be stated for reproducibility.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: QURE's central claims are evaluated on external benchmarks, and the hard-negative rule is a training heuristic rather than a definitional restatement of the reported results.

full rationale

The paper's load-bearing components are not circular. The hard-negative set H in Eq. (7) is built by sorting corpus images with the in-training relevance score s from Eq. (1) and selecting the range between the two largest score drops after the target. This is a self-referential training loop, but no reported result is derived from H by construction: FashionIQ, CIRR, and CIRCO metrics are computed against held-out labels, and the HP-FashionIQ preference labels were annotated by humans independently of QURE's scores. The training objective in Eq. (3) is a standard Bradley-Terry pairwise ranking loss; afterward, the paper measures whether the same score ordering agrees with human preferences, which is an external alignment test, not a fitted prediction. The claim that 'steep drops indicate significant changes in relevance' is supported by an external citation (Xia et al., 2024), not by the authors' own prior work, so no self-citation chain is load-bearing. The main weakness is that the boundary assumption—that the two largest score drops after the target separate false negatives from true hard negatives—is not validated against ground-truth relevance; Figure 6 shows only two qualitative examples and Figure 8 shows aggregate score curves without labels confirming the boundary. That is a correctness and robustness concern, not circularity, because the method's success is still judged by external benchmarks. A separate experimental inconsistency exists in that HP-FashionIQ pairs were generated from only four baseline models, yet QURE is scored on the dataset, but this is a data-validity issue outside the circularity definition. Overall, the derivation chain is self-contained against external benchmarks, so the circularity score is 0.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central algorithm depends on standard ML components: pretrained BLIP-2 features, a Bradley-Terry pairwise loss, and a self-referential hard-negative mining rule. The hand-chosen design choices are ndef=6, one sampled negative per query per epoch, and the top-2 drop rule. No new physical entities or conceptual constructs are proposed; the new HP-FashionIQ dataset is a measurement resource, not an invented entity.

free parameters (3)
  • ndef = 6
    Hard negative set is recomputed every floor(nepoch/ndef) epochs; set to 6 for both FashionIQ and CIRR with no reported ablation or sensitivity analysis.
  • one negative sampled per query per epoch = 1
    Algorithm 1 samples a single hard negative per query per epoch uniformly from the hard negative set; this choice is not ablated.
  • top-2 largest relevance drops = 2
    Eq. (6) uses the top two largest relevance degradations after the target to define the hard negative interval; no threshold or ablation on this k is reported.
assumptions (5)
  • domain assumption Bradley-Terry model in Eq. (2) describes user preference between images for composed image queries.
    The training objective treats target-over-hard-negative preference as logistic score differences; this is standard in reward modeling but is imported into CIR without justification.
  • ad hoc to paper After the warm-up phase, the model's relevance scores are accurate enough to identify false negatives and hard negatives.
    Section 3.2 sorts the corpus by model scores and relies on those scores to find the two largest drops; no ground-truth relevance labels validate this.
  • ad hoc to paper The two largest relevance-score drops after the target separate false negatives, hard negatives, and easy negatives.
    Eq. (6)-(7) defines the hard negative set from these drops; the paper relies on a citation to a medical RAG paper and on qualitative examples rather than a CIR-specific validation.
  • domain assumption BLIP-2's pretrained image encoder and Q-Former provide adequate cross-modal relevance features for CIR.
    The relevance score in Eq. (1) is built on BLIP-2; all results are conditional on this backbone.
  • domain assumption Human annotations in HP-FashionIQ reflect true user satisfaction with retrieved sets.
    The dataset uses 61 participants and excludes 335 queries; annotation noise and exclusion criteria can bias the preference labels.

how reviews work

0 comments
Cite this review

Pith. "Pith review of QuRe: Query-Relevant Retrieval through Hard Negative Sampling in Composed Image Retrieval." pith.science (2026). https://pith.science/paper/UNHQLDO5

@misc{pith2026250712416,
  author       = {Pith},
  title        = {Pith review of: QuRe: Query-Relevant Retrieval through Hard Negative Sampling in Composed Image Retrieval},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UNHQLDO5}},
  note         = {Machine review of arXiv:2507.12416}
}
read the original abstract

Composed Image Retrieval (CIR) retrieves relevant images based on a reference image and accompanying text describing desired modifications. However, existing CIR methods only focus on retrieving the target image and disregard the relevance of other images. This limitation arises because most methods employing contrastive learning-which treats the target image as positive and all other images in the batch as negatives-can inadvertently include false negatives. This may result in retrieving irrelevant images, reducing user satisfaction even when the target image is retrieved. To address this issue, we propose Query-Relevant Retrieval through Hard Negative Sampling (QuRe), which optimizes a reward model objective to reduce false negatives. Additionally, we introduce a hard negative sampling strategy that selects images positioned between two steep drops in relevance scores following the target image, to effectively filter false negatives. In order to evaluate CIR models on their alignment with human satisfaction, we create Human-Preference FashionIQ (HP-FashionIQ), a new dataset that explicitly captures user preferences beyond target retrieval. Extensive experiments demonstrate that QuRe achieves state-of-the-art performance on FashionIQ and CIRR datasets while exhibiting the strongest alignment with human preferences on the HP-FashionIQ dataset. The source code is available at https://github.com/jackwaky/QuRe.

Figures

Figures reproduced from arXiv: 2507.12416 by the authors.

Figure 1
Figure 1. Comparison of existing CIR methods and QURE. Tradi￾tional CIR approaches treat all non-target images as negatives in contrastive learning. In contrast, QURE ranks the image corpus using a learned relevance score to identify a hard negative set. It then applies preference-based optimization to distinguish not only the target image but also other relevant images from hard negatives, leading to improved retrieval perfo… view at source ↗
Figure 2
Figure 2. Overview of QURE. During training, QURE periodically ranks corpus images by relevance score using the current model. Hard negatives are selected from the range between two sharp drops in relevance scores following the target image. A KL divergence loss is then used to train the model to assign higher relevance to the positive image than to a randomly chosen hard negative. including false negatives as negatives, whic… view at source ↗
Figure 3
Figure 3. An example from the HP-FashionIQ dataset: Given a query, two retrieved image sets are presented. In this example, the user preferred Set 2, as it better preserves the visual properties of the reference image. Modality redundancy check. While CIR should consider both image and text, some examples focus solely on the image or the text. CASE (Levy et al., 2024) highlighted modality redundancy in FashionIQ, indicating t… view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Average recall on the FashionIQ validation set using four hard negative set definitions. After the initial hard negative set is established (e.g., at epoch 4), the choice of definition significantly influences average recall throughout training. demonstrates QURE’s abi…
Figure 6
Figure 6. Figure 6: Hard negative set examples for two shirt-category queries in FashionIQ. Each set contains images that are semantically similar to the target but deemed less relevant by the model. timately leads to suboptimal performance as the selected negatives from the entire corpus…
Figure 7
Figure 7. Figure 7: Visualization of sorted relevance scores for a FashionIQ query at two stages: prior to training and after completing the warm-up phase. C. HP-FashionIQ Dataset C.1. Data Annotation Examples We conducted a data collection via Google Forms. Each form consisted of instruc…
Figure 8
Figure 8. Figure 8: Visualization of sorted relevance scores aggregated over all FashionIQ queries at two stages: before training and after the warm-up phase [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: shows the guidelines provided to participants, who were asked to score the retrieved image sets based on relevance to the query using a 5-point Likert scale. We gave participants a relatable scenario that required them to evaluate the results from two different online …
Figure 10
Figure 10. Figure 10: Example of query and two set of retrieved images in user survey. 14 [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]
Figure 11
Figure 11. Figure 11: Example of questions in user survey. 15 [PITH_FULL_IMAGE:figures/full_fig_p015_11.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

37 extracted references · 20 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    and Sanderson, M

    Al-Maskari, A. and Sanderson, M. A review of factors influencing user satisfaction in information retrieval. Journal of the American Society for Information Science and Technology, 61 0 (5): 0 859--868, 2010

    work page 2010

  3. [3]

    Flamingo: a visual language model for few-shot learning

    Alayrac, J.-B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., et al. Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems, 35: 0 23716--23736, 2022

    2022

  4. [4]

    Qwen technical report

    Bai, J., Bai, S., Chu, Y., Cui, Z., Dang, K., Deng, X., Fan, Y., Ge, W., Han, Y., Huang, F., et al. Qwen technical report. arXiv preprint arXiv:2309.16609, 2023

    arXiv 2023

  5. [5]

    Bai, Y., Xu, X., Liu, Y., Khan, S., Khan, F., Zuo, W., Goh, R. S. M., and Feng, C.-M. Sentence-level prompts benefit composed image retrieval. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=m3ch3kJL7q

    work page 2024

  6. [6]

    Composed image retrieval using contrastive learning and task-oriented clip-based features

    Baldrati, A., Bertini, M., Uricchio, T., and Del Bimbo, A. Composed image retrieval using contrastive learning and task-oriented clip-based features. ACM Transactions on Multimedia Computing, Communications and Applications, 20 0 (3): 0 1--24, 2023

    work page 2023

  7. [7]

    Bradley, R. A. and Terry, M. E. Rank Analysis of Incomplete Block Designs: The Method of Paired Comparisons . Biometrika, 39 0 (3-4): 0 324--345, 12 1952. ISSN 0006-3444. doi:10.1093/biomet/39.3-4.324. URL https://doi.org/10.1093/biomet/39.3-4.324

    doi:10.1093/biomet/39.3-4.324 1952

  8. [8]

    Composed image retrieval with text feedback via multi-grained uncertainty regularization

    Chen, Y., Zheng, Z., Ji, W., Qu, L., and Chua, T.-S. Composed image retrieval with text feedback via multi-grained uncertainty regularization. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=Yb5KvPkKQg

    work page 2024

Show all 37 references
  1. [9]

    Modality-agnostic attention fusion for visual search with text feedback

    Dodds, E., Culpepper, J., Herdade, S., Zhang, Y., and Boakye, K. Modality-agnostic attention fusion for visual search with text feedback. arXiv preprint arXiv:2007.00145, 2020

    2007 arXiv

  2. [10]

    Improving composed image retrieval via contrastive learning with scaling positives and negatives

    Feng, Z., Zhang, R., and Nie, Z. Improving composed image retrieval via contrastive learning with scaling positives and negatives. arXiv preprint arXiv:2404.11317, 2024

    2024 arXiv

  3. [11]

    R., Maire, M., and Khademi, M

    Huynh, T., Kornblith, S., Walter, M. R., Maire, M., and Khademi, M. Boosting contrastive self-supervised learning with false negative cancellation. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp.\ 2785--2795, 2022

    2022

  4. [12]

    Scaling up visual and vision-language representation learning with noisy text supervision

    Jia, C., Yang, Y., Xia, Y., Chen, Y.-T., Parekh, Z., Pham, H., Le, Q., Sung, Y.-H., Li, Z., and Duerig, T. Scaling up visual and vision-language representation learning with noisy text supervision. In International conference on machine learning, pp.\ 4904--4916. PMLR, 2021

    2021

  5. [13]

    Cosmo: Content-style modulation for image retrieval with text feedback

    Lee, S., Kim, D., and Han, B. Cosmo: Content-style modulation for image retrieval with text feedback. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 802--812, 2021

    2021

  6. [14]

    Data roaming and quality assessment for composed image retrieval

    Levy, M., Ben-Ari, R., Darshan, N., and Lischinski, D. Data roaming and quality assessment for composed image retrieval. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp.\ 2991--2999, 2024

    2024

  7. [15]

    Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation

    Li, J., Li, D., Xiong, C., and Hoi, S. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In International conference on machine learning, pp.\ 12888--12900. PMLR, 2022

    2022

  8. [16]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models

    Li, J., Li, D., Savarese, S., and Hoi, S. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning, pp.\ 19730--19742. PMLR, 2023

    2023

  9. [17]

    H., Yatskar, M., Yin, D., Hsieh, C.-J., and Chang, K.-W

    Li, L. H., Yatskar, M., Yin, D., Hsieh, C.-J., and Chang, K.-W. Visualbert: A simple and performant baseline for vision and language. arXiv preprint arXiv:1908.03557, 2019

    1908 arXiv

  10. [18]

    Improving context understanding in multimodal large language models via multimodal composition learning

    Li, W., Fan, H., Wong, Y., Yang, Y., and Kankanhalli, M. Improving context understanding in multimodal large language models via multimodal composition learning. In Forty-first International Conference on Machine Learning

  11. [19]

    A technique for the measurement of attitudes

    Likert, R. A technique for the measurement of attitudes. Archives of psychology, 1932

    1932

  12. [20]

    Image retrieval on real-life images with pre-trained vision-and-language models

    Liu, Z., Rodriguez-Opazo, C., Teney, D., and Gould, S. Image retrieval on real-life images with pre-trained vision-and-language models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp.\ 2125--2134, 2021

    2021

  13. [21]

    Candidate set re-ranking for composed image retrieval with dual multi-modal encoder

    Liu, Z., Sun, W., Teney, D., and Gould, S. Candidate set re-ranking for composed image retrieval with dual multi-modal encoder. arXiv preprint arXiv:2305.16304, 2023

    2023 arXiv

  14. [22]

    Bi-directional training for composed image retrieval via text prompt learning

    Liu, Z., Sun, W., Hong, Y., Teney, D., and Gould, S. Bi-directional training for composed image retrieval via text prompt learning. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp.\ 5753--5762, 2024

    2024

  15. [23]

    Decoupled weight decay regularization

    Loshchilov, I. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017

    2017 arXiv

  16. [24]

    Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks

    Lu, J., Batra, D., Parikh, D., and Lee, S. Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks. Advances in neural information processing systems, 32, 2019

    2019

  17. [25]

    Active contrastive learning of audio-visual video representations

    Ma, S., Zeng, Z., McDuff, D., and Song, Y. Active contrastive learning of audio-visual video representations. arXiv preprint arXiv:2009.09805, 2020

    2009 arXiv

  18. [26]

    Training language models to follow instructions with human feedback

    Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35: 0 27730--27744, 2022

    2022

  19. [27]

    W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al

    Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp.\ 8748--8763. PMLR, 2021

    2021

  20. [28]

    Contrastive learning with hard negative samples

    Robinson, J., Chuang, C.-Y., Sra, S., and Jegelka, S. Contrastive learning with hard negative samples. arXiv preprint arXiv:2010.04592, 2020

    2010 arXiv

  21. [29]

    Pic2word: Mapping pictures to words for zero-shot composed image retrieval

    Saito, K., Sohn, K., Zhang, X., Li, C.-L., Lee, C.-Y., Saenko, K., and Pfister, T. Pic2word: Mapping pictures to words for zero-shot composed image retrieval. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.\ 19305--19314, 2023

    2023

  22. [30]

    A corpus for reasoning about natural language grounded in photographs

    Suhr, A., Zhou, S., Zhang, A., Zhang, I., Bai, H., and Artzi, Y. A corpus for reasoning about natural language grounded in photographs. arXiv preprint arXiv:1811.00491, 2018

    2018 arXiv

  23. [31]

    Hard negative sampling strategies for contrastive representation learning

    Tabassum, A., Wahed, M., Eldardiry, H., and Lourentzou, I. Hard negative sampling strategies for contrastive representation learning. arXiv preprint arXiv:2206.01197, 2022

    2022 arXiv

  24. [32]

    Covr-2: Automatic data construction for composed video retrieval

    Ventura, L., Yang, A., Schmid, C., and Varol, G. Covr-2: Automatic data construction for composed video retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024 a

    2024

  25. [33]

    CoVR : Learning composed video retrieval from web video captions

    Ventura, L., Yang, A., Schmid, C., and Varol, G. CoVR : Learning composed video retrieval from web video captions. AAAI, 2024 b

    2024

  26. [34]

    Composing text and image for image retrieval-an empirical odyssey

    Vo, N., Jiang, L., Sun, C., Murphy, K., Li, L.-J., Fei-Fei, L., and Hays, J. Composing text and image for image retrieval-an empirical odyssey. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.\ 6439--6448, 2019

    2019

  27. [35]

    Fashion iq: A new dataset towards retrieving images by natural language feedback

    Wu, H., Gao, Y., Guo, X., Al-Halah, Z., Rennie, S., Grauman, K., and Feris, R. Fashion iq: A new dataset towards retrieving images by natural language feedback. In Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pp.\ 11307--11317, 2021

    2021

  28. [36]

    Mmed-rag: Versatile multimodal rag system for medical vision language models

    Xia, P., Zhu, K., Li, H., Wang, T., Shi, W., Wang, S., Zhang, L., Zou, J., and Yao, H. Mmed-rag: Versatile multimodal rag system for medical vision language models. arXiv preprint arXiv:2410.13085, 2024

    2024 arXiv

  29. [37]

    Magiclens: Self-supervised image retrieval with open-ended instructions

    Zhang, K., Luan, Y., Hu, H., Lee, K., Qiao, S., Chen, W., Su, Y., and Chang, M.-W. Magiclens: Self-supervised image retrieval with open-ended instructions. Forty-first International Conference on Machine Learning, 2024

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.