Pith. sign in

REVIEW 4 major objections 6 minor 43 references

SORCE: Small Object Retrieval in Complex Environments

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Multiple embeddings beat single-feature retrieval for small objects

desk verdict Useful new benchmark, but the paper's own Table 4 leaves the method's causal contribution unproven — the gain could be from cropping or fine-tuning, not ReP. read the letter →

arxiv 2505.24441 v1 pith:JNIGCWAD submitted 2025-05-30 cs.CV

Chunxu Liu , Chi Xie , Xiaxu Chen , Wei Li , Feng Zhu , Rui Zhao , Limin Wang This is my paper
classification cs.CV
keywords smallobjectretrievaltext-to-imageSORCE-1Kbenchmarkmultimodallargelanguagemodelsregionalpromptsmulti-embeddingrepresentationcontrastivefine-tuningcomplexscene
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces SORCE (Small Object Retrieval in Complex Environments), a new subfield of text-to-image retrieval aimed at finding a small, inconspicuous object described by text inside a cluttered image. To study it, the authors build SORCE-1K, a benchmark of 1,023 high-resolution scene images with captions that describe only the small target and minimal context. They report that current retrieval methods, which compress each image into one embedding, perform poorly on this benchmark, and they propose instead to represent each image by several region-focused embeddings extracted from a Multimodal Large Language Model with Regional Prompts. Their experiments show this multi-embedding method raises Full Res. R@1 on SORCE-1K from 21.9 to 27.7, and to 31.5 after contrastive fine-tuning, without harming standard Flickr30K and COCO retrieval. If correct, this establishes multi-embedding MLLM features as a simple baseline for small-object retrieval.

What carries the argument

Regional Prompts (ReP): a small fixed set of natural-language instructions that ask an MLLM to summarize a specific quadrant of the image (left upper, right upper, left lower, right lower) plus a global summary, each in one word. The MLLM's final-token embedding under each prompt yields a distinct feature for the same image, and the paper shows these features align with independently extracted features of the corresponding cropped regions while still retaining global information. Contrastive fine-tuning (with random choice of one regional prompt per training step and the COCO-118K captions recaptioned into regional descriptions by InternVL2.5-38B) sharpens the text-image alignment.

What would settle it

Run an ablation on SORCE-1K in which the small target region is occluded or blurred before retrieval while the query is unchanged; if R@1 stays near the original value, the model is retrieving from background context, not the small object. A second check is to add distractors that reuse the same context words (same scene type, similar colors) but describe a different small object; if retrieval accuracy collapses, the descriptions leak context and the benchmark is measuring caption disambiguation.

Watch

Extended reading notes

Core claim

The central claim is that representing an image with multiple MLLM embeddings, each elicited by a fixed regional prompt ('summarize the left upper corner of the image in one word', etc.), captures the small, non-salient target that a single global embedding loses. The paper demonstrates this on the new SORCE-1K benchmark and shows the gain is not just ensembling: synonym prompts for the global summary give negligible improvement, while regional prompts give a large one. The paper further claims that contrastive fine-tuning, aligning randomly chosen regional-prompt features with recaptioned regional text, enhances the effect, and that cropping the image into patches is inferior because it discards global context.

Load-bearing premise

The load-bearing premise is that the hand-written SORCE-1K descriptions are uniquely matched to one target and contain no contextual cues that let a model find the image without perceiving the small object itself, with manual review as the only safeguard.

Editorial extensions

If this is right

  • Existing single-embedding retrieval models, CLIP and MLLM alike, underperform markedly on small-object queries in complex scenes, so SORCE-1K can serve as a diagnostic for that failure mode.
  • Five region-prompted features per image are a simple improvement over one global feature, giving a strong baseline for future SORCE methods.
  • Contrastive fine-tuning with randomly selected regional prompts improves the alignment further, at a modest training cost of one epoch on COCO-118K.
  • The three zoom tiers make SORCE-1K usable to analyze how retrieval degrades as object size shrinks relative to context.
  • Because performance on Flickr30K and COCO stays comparable, the multi-embedding method does not sacrifice standard retrieval quality to gain small-object performance.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If SORCE-1K has any hidden contextual shortcuts, the published gains could overstate the models' perceptual access to tiny objects; a quick check would be to compare retrieval with the target box removed, which the paper does not report.
  • The ReP scheme suggests a general recipe for fine-grained retrieval: fix a set of spatial or semantic viewpoints, prompt an MLLM per viewpoint, and keep the closest feature at query time. This could transfer to instance-level retrieval or described-object detection with little change.
  • Storing five features per image multiplies memory; an implicit extension is to let the query select which regional feature to use, or to have the MLLM adaptively emit only the needed number of features.
  • The benchmark's three difficulty levels could expose scale-dependent failure modes in CLIP-style models, guiding future work on resolution-aware encoders.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces SORCE (Small Object Retrieval in Complex Environments), a new text-to-image retrieval setting focused on retrieving images containing small, non-salient objects in cluttered scenes. The authors construct SORCE-1K, an evaluation-only benchmark of 1,023 images from SA-1B, with manual regional descriptions designed to uniquely identify a small target with minimal contextual cues. They propose representing each image with multiple embeddings obtained from an MLLM (E5-V) using a set of Regional Prompts (ReP) plus a global prompt, selecting the closest feature for retrieval. A contrastive fine-tuning stage on recaptioned COCO-118K aligns regional features with regional text descriptions. Experiments on SORCE-1K show gains over single-embedding baselines (e.g., Full Res. R@1 from 21.9 for E5-V to 31.5 for E5-V (ft.) + ReP), while performance on Flickr30K and COCO remains comparable.

Significance. If SORCE-1K is a valid benchmark, it fills a genuine gap: existing T2IR benchmarks predominantly test holistic or salient-object descriptions, and small-object retrieval in complex scenes is relevant to surveillance, infrastructure, and consumer search. The multi-embedding idea is simple, practical, and partially supported by MLLM behavior; the paper also ships a benchmark, difficulty tiers (Full Res., Zoom-2x, Zoom-3x), and ablations (synonym prompts, prompt count, cropping) that provide useful controls. The external validation on Flickr30K/COCO is a strength, as is the attempt to minimize contextual leakage in queries. However, the central causal claim that Regional Prompts and MLLM-based text-guided features drive the improvement is weakened by the paper's own 2x2-split ablation, which performs comparably or better on the hardest setting before fine-tuning, and the fine-tuned 2x2-split baseline is not reported. The benchmark's uniqueness guarantee also rests on manual review without inter-annotator agreement or negative-query evaluation, which leaves room for hidden contextual shortcuts.

major comments (4)
  1. [§5.3, Table 4] The paper's own ablation undercuts the central claim that ReP is the effective component. Table 4 shows that E5-V + 2x2 Split reaches Full Res. R@5 = 47.9, above E5-V + ReP's 45.4, and the text states that cropping 'surpasses regional prompts.' Since the headline gain over single-embedding baselines could come from (a) any multi-embedding max-pooling, (b) contrastive fine-tuning, or (c) ReP specifically, the missing (ft.) + 2x2 Split row is a load-bearing omission. To support the paper's conclusion, the authors should report fine-tuned 2x2-split features under the same protocol, and ideally also report R@1/R@10 for that baseline. If the fine-tuned split also matches or exceeds (ft.) + ReP, the claimed advantage of MLLM-based regional prompts over generic spatial splits is not established.
  2. [§3.2, Figure 4] The benchmark's validity depends on the assumption that each manually written description uniquely identifies one target image with no reliance on surrounding context. The paper's only guarantee is manual review by the authors; there is no inter-annotator agreement, no negative-query or adversarial-query evaluation, and no external audit. Because retrieval metrics are computed over the whole gallery, any description that accidentally contains a distinctive background/context cue could be solved by shortcut matching rather than small-object perception. The authors should provide additional validation, for example: (i) measure retrieval with the query paired against target-only crops versus full images, (ii) test queries with the target region masked or with distractor images sharing similar context, or (iii) report human agreement on whether a description uniquely identifies the intended object in the full gallery.
  3. [§5.2, Table 1] The paper uses 'significantly outperforms' in Sec. 1 and the abstract, but no error bars, repeated runs, or significance tests are reported anywhere. Differences between closely related variants (e.g., Table 10: E5-V (ft.) + Semantic Prompts Full Res. R@1 = 28.0 vs E5-V (ft.) + ReP = 31.5) are the basis for a selective claim about ReP, yet with a 1,023-image gallery, such gaps need confidence intervals or bootstrap estimates to be interpretable. Please add at least bootstrapped 95% confidence intervals for the main SORCE-1K results, or multiple fine-tuning runs with different seeds.
  4. [§4.2, Eq. (1)-(2)] The contrastive fine-tuning recaptions COCO-118K with InternVL2.5-38B, using the same MLLM family that E5-V is derived from (LLaVA-NeXT-8B). While this is not circular in the strict sense, the training captions are model-generated and may carry the same regional bias as the inference prompts. Please discuss or ablate the choice of the recaptioning model (e.g., a strong captioner outside the LLaVA family) to ensure the fine-tuning gain is not an artifact of self-generated captions.
minor comments (6)
  1. [§4.1] There is a typo: 'SCORCE-1K' should be 'SORCE-1K'.
  2. [§6] In the conclusion, 'text-custimized' should be 'text-customized'.
  3. [References] The reference list appears to duplicate Long-CLIP: [39] and [40] both refer to the same paper, and the in-text citation for Urban-1K [40] in Sec. 2.1 does not match a distinct reference. Please correct the numbering and ensure each cited work has a unique entry.
  4. [Table 5] Table 5 reports only R@5 for Flickr and COCO; including the full R@1/R@10 rows would make the comparison with Table 2 more transparent, since the conclusion about cropping's degradation is based on a single metric.
  5. [§5.3, Table 4] The text says the 2x2 split baseline uses five features including the global feature, but it is not explicitly stated whether the five regional-prompt features also include the global summary prompt; clarifying this would make the comparison exactly matched.
  6. [Figure 5] The t-SNE plot in Fig. 5c is small and hard to read; consider enlarging and labeling the clusters, or providing a quantitative measure (e.g., mean distance to global feature) to support the qualitative claim.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the SORCE-1K results are empirical, the only self-citation is a non-load-bearing related-work pointer, and the central comparisons on Flickr30K and COCO are external.

full rationale

The paper's central claim that multi-embedding representation through MLLM and Regional Prompts improves small-object retrieval is supported by experiments on the authors' own SORCE-1K benchmark and by comparisons on external benchmarks (Flickr30K, COCO). No equation or construction step defines the reported retrieval scores in terms of the method's inputs. The method is not fitted on SORCE-1K and then evaluated on SORCE-1K; contrastive fine-tuning is performed on COCO-118K, a separate dataset. The only self-citation is Ref. [34] in Section 3.3, where the authors mention described object detection as an additional application of their benchmark; that citation is incidental and does not support any load-bearing premise. The internal ablation gap concerning 2x2 split features (Table 4) is a potential attribution or experimental-design weakness, but it is not circularity: the paper does not claim, by construction, that ReP must beat cropping, and the reported numbers remain empirical outcomes. Therefore, the derivation chain is self-contained with respect to the formal patterns of circularity considered here.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central empirical claims rest on benchmark construction choices and on properties of MLLM embeddings that are asserted rather than independently established. No new physical or conceptual entities are introduced.

free parameters (3)
  • Temperature tau in contrastive loss = not reported
    Eq. 2 in Sec. 4.2 scales logits in the cross-entropy loss; its value affects learned embedding geometry and downstream retrieval, but the paper does not state it.
  • Number of regional prompts (five) = 5 (four regional + one global)
    Chosen by the authors; Tab. 6 shows performance rises monotonically with prompt count, so the fixed number is a hand-selected design choice that directly affects reported SORCE-1K scores.
  • Benchmark filtering thresholds = bbox area <=20%, caption >=8 words, target area usually <10%
    These thresholds in Sec. 3.2 and Appendix C define which images enter SORCE-1K and therefore shape every evaluation number; they are chosen by the authors, not derived from an external standard.
assumptions (4)
  • domain assumption SA-1B mask count is a valid proxy for environmental complexity
    Sec. 3.1 and Appendix A assume scenes with more segmentation masks are more complex; this is plausible but not independently validated, and it guides image selection for SORCE-1K.
  • domain assumption MLLM last-token embeddings can be steered by regional prompts to focus on named image regions while retaining global context
    Sec. 4.1 supports this with qualitative similarity matrices and t-SNE on 20 images, but the core mechanism is not proven; the method and ablations rely on it.
  • domain assumption The manually written descriptions in SORCE-1K are unique and contain minimal contextual shortcuts
    Sec. 3.2 asserts manual review guarantees this, but no inter-annotator agreement, adversarial query test, or external check is reported; retrieval scores on the benchmark depend on this.
  • domain assumption Contrastive fine-tuning on InternVL2.5-38B recaptioned COCO-118K improves alignment with human-style regional queries
    Sec. 4.2 assumes synthetic regional captions from an MLLM are a good training signal; this is not tested against human-annotated regional descriptions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SORCE: Small Object Retrieval in Complex Environments." pith.science (2026). https://pith.science/paper/JNIGCWAD

@misc{pith2026250524441,
  author       = {Pith},
  title        = {Pith review of: SORCE: Small Object Retrieval in Complex Environments},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JNIGCWAD}},
  note         = {Machine review of arXiv:2505.24441}
}
read the original abstract

Text-to-Image Retrieval (T2IR) is a highly valuable task that aims to match a given textual query to images in a gallery. Existing benchmarks primarily focus on textual queries describing overall image semantics or foreground salient objects, possibly overlooking inconspicuous small objects, especially in complex environments. Such small object retrieval is crucial, as in real-world applications, the targets of interest are not always prominent in the image. Thus, we introduce SORCE (Small Object Retrieval in Complex Environments), a new subfield of T2IR, focusing on retrieving small objects in complex images with textual queries. We propose a new benchmark, SORCE-1K, consisting of images with complex environments and textual queries describing less conspicuous small objects with minimal contextual cues from other salient objects. Preliminary analysis on SORCE-1K finds that existing T2IR methods struggle to capture small objects and encode all the semantics into a single embedding, leading to poor retrieval performance on SORCE-1K. Therefore, we propose to represent each image with multiple distinctive embeddings. We leverage Multimodal Large Language Models (MLLMs) to extract multiple embeddings for each image instructed by a set of Regional Prompts (ReP). Experimental results show that our multi-embedding approach through MLLM and ReP significantly outperforms existing T2IR methods on SORCE-1K. Our experiments validate the effectiveness of SORCE-1K for benchmarking SORCE performances, highlighting the potential of multi-embedding representation and text-customized MLLM features for addressing this task.

Figures

Figures reproduced from arXiv: 2505.24441 by the authors.

Figure 1
Figure 1. Instances from different retrieval benchmarks. We draw the bounding boxes of the referred objects in the image. tampered watermarks and manipulated content; in consumer applications, it enables users to locate specific small objects within personal photo collections efficiently. Despite the importance of small object retrieval, existing T2IR benchmarks primarily focus on retrieving images based on holistic descripti… view at source ↗
Figure 2
Figure 2. The construction process for SORCE-1K. Zoom-2× Zoom-3× [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Example of the difficulty levels. We perform zooming 2× and 3× and crop the image while ensuring the object is in the resulting frame. More qualitative examples in Sec. D. 3 SORCE-1K Benchmark 3.1 Dataset Highlights and Statistics The proposed benchmark follows the same task formulation and evaluation protocol as existing text-to-image retrieval benchmarks. However, its key challenges differ from traditional retriev… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Statistics for the proposed SORCE-1K benchmark. the high-resolution image. As is shown in [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Qualitative comparisons with regional prompt generated features and independently generated features from 2 × 2 splits. 4 Method 4.1 Multiple Feature Representation As shown in Tab. 1, existing approaches obtain poor results on the Full Res. setting of SCORCE-1K. This …
Figure 6
Figure 6. Figure 6: Pipeline of our contrastive finetuning process. Random Choice block means that we randomly choose a regional prompt from the right, and use the resulting image embedding to align with the text description for the corresponding regional regions. To employ regional promp…
Figure 7
Figure 7. Figure 7: Mask distribution of SORCE-1K. Environment complexity usually grows with the number of segmentation masks. B The Caption Length Distribution of Flickr30K and COCO We present the caption length distribution charts in [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]
Figure 8
Figure 8. Figure 8: Caption length distribution charts of Flickr30K and COCO. 13 [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: More visualization examples from SORCE-1K benchmark. 15 [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

43 extracted references · 18 canonical work pages

  1. [1]

    X-detr: A versatile architecture for instance-wise vision-language tasks

    Zhaowei Cai, Gukyeong Kwon, Avinash Ravichandran, Erhan Bas, Zhuowen Tu, Rahul Bhotika, and Stefano Soatto. X-detr: A versatile architecture for instance-wise vision-language tasks. In European Conference on Computer Vision, pages 290–308. Springer, 2022

    work page 2022

  2. [2]

    Sharegpt4v: Improving large multi-modal models with better captions

    Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Conghui He, Jiaqi Wang, Feng Zhao, and Dahua Lin. Sharegpt4v: Improving large multi-modal models with better captions. InEuropean Conference on Computer Vision, pages 370–387. Springer, 2024

    2024

  3. [3]

    Learning local similarity with spatial relations for object retrieval

    Zhenfang Chen, Zhanghui Kuang, Wayne Zhang, and Kwan-Yee K Wong. Learning local similarity with spatial relations for object retrieval. InProceedings of the 27th ACM international conference on Multimedia, pages 1703–1711, 2019

    work page 2019

  4. [4]

    Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling.arXiv preprint arXiv:2412.05271, 2024

    Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shen- glong Ye, Hao Tian, Zhaoyang Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling.arXiv preprint arXiv:2412.05271, 2024

    arXiv 2024

  5. [5]

    Qlora: Efficient finetuning of quantized llms.Advances in neural information processing systems, 36:10088– 10115, 2023

    Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. Qlora: Efficient finetuning of quantized llms.Advances in neural information processing systems, 36:10088– 10115, 2023

    2023

  6. [6]

    Mme: A comprehensive evaluation benchmark for multimodal large language models.arXiv preprint arXiv:2306.13394, 2023

    Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, et al. Mme: A comprehensive evaluation benchmark for multimodal large language models.arXiv preprint arXiv:2306.13394, 2023

    arXiv 2023

  7. [7]

    Efficient diffusion on region manifolds: Recovering small objects with compact cnn representations

    Ahmet Iscen, Giorgos Tolias, Yannis Avrithis, Teddy Furon, and Ondrej Chum. Efficient diffusion on region manifolds: Recovering small objects with compact cnn representations. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 2077–2086, 2017

    2017

  8. [8]

    Scaling sen- tence embeddings with large language models

    Ting Jiang, Shaohan Huang, Zhongzhi Luan, Deqing Wang, and Fuzhen Zhuang. Scaling sen- tence embeddings with large language models. InFindings of the Association for Computational Linguistics: EMNLP 2024, pages 3182–3196, 2024

    2024

Show all 43 references
  1. [9]

    E5-v: Universal embeddings with multimodal large language models.arXiv preprint arXiv:2407.12580, 2024

    Ting Jiang, Minghui Song, Zihan Zhang, Haizhen Huang, Weiwei Deng, Feng Sun, Qi Zhang, Deqing Wang, and Fuzhen Zhuang. E5-v: Universal embeddings with multimodal large language models.arXiv preprint arXiv:2407.12580, 2024

    2024 arXiv

  2. [10]

    Seed-bench: Benchmarking multimodal large language models

    Bohao Li, Yuying Ge, Yixiao Ge, Guangzhi Wang, Rui Wang, Ruimao Zhang, and Ying Shan. Seed-bench: Benchmarking multimodal large language models. InCVPR, pages 13299–13308, 2024

    2024

  3. [11]

    Llava-next: What else influences visual instruction tuning beyond data?, 2024

    Bo Li, Hao Zhang, Kaichen Zhang, Dong Guo, Yuanhan Zhang, Renrui Zhang, Feng Li, Ziwei Liu, and Chunyuan Li. Llava-next: What else influences visual instruction tuning beyond data?, 2024

    2024

  4. [12]

    Llava-next: Stronger llms supercharge multimodal capabilities in the wild, 2024

    Bo Li, Kaichen Zhang, Hao Zhang, Dong Guo, Renrui Zhang, Feng Li, Yuanhan Zhang, Ziwei Liu, and Chunyuan Li. Llava-next: Stronger llms supercharge multimodal capabilities in the wild, 2024

    2024

  5. [13]

    Percep- tual generative adversarial networks for small object detection

    Jianan Li, Xiaodan Liang, Yunchao Wei, Tingfa Xu, Jiashi Feng, and Shuicheng Yan. Percep- tual generative adversarial networks for small object detection. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 1222–1230, 2017

    2017

  6. [14]

    MM-EMBED: UNIVERSAL MULTIMODAL RETRIEV AL WITH MULTIMODAL LLMS

    Sheng-Chieh Lin, Chankyu Lee, Mohammad Shoeybi, Jimmy Lin, Bryan Catanzaro, and Wei Ping. MM-EMBED: UNIVERSAL MULTIMODAL RETRIEV AL WITH MULTIMODAL LLMS. InThe Thirteenth International Conference on Learning Representations, 2025

    2025

  7. [15]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. InComputer vision–ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceedings,...

    2014

  8. [16]

    Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023

    2023

  9. [17]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26296–26306, 2024

    2024

  10. [18]

    Lamra: Large multimodal model as your advanced retrieval assistant.arXiv preprint arXiv:2412.01720, 2024

    Yikun Liu, Pingan Chen, Jiayin Cai, Xiaolong Jiang, Yao Hu, Jiangchao Yao, Yanfeng Wang, and Weidi Xie. Lamra: Large multimodal model as your advanced retrieval assistant.arXiv preprint arXiv:2412.01720, 2024

    2024 arXiv

  11. [19]

    Mmbench: Is your multi-modal model an all-around player? InEuropean conference on computer vision, pages 216–233

    Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, et al. Mmbench: Is your multi-modal model an all-around player? InEuropean conference on computer vision, pages 216–233. Springer, 2024

    2024

  12. [20]

    Image retrieval on real-life images with pre-trained vision-and-language models

    Zheyuan Liu, Cristian Rodriguez-Opazo, Damien Teney, and Stephen Gould. Image retrieval on real-life images with pre-trained vision-and-language models. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 2125–2134, 2021

    2021

  13. [21]

    Scaling open-vocabulary object detection.Advances in Neural Information Processing Systems, 36:72983–73007, 2023

    Matthias Minderer, Alexey Gritsenko, and Neil Houlsby. Scaling open-vocabulary object detection.Advances in Neural Information Processing Systems, 36:72983–73007, 2023

    2023

  14. [22]

    Better to follow, follow to be better: Towards precise supervision of feature super-resolution for small object detection

    Junhyug Noh, Wonho Bae, Wonhee Lee, Jinhwan Seo, and Gunhee Kim. Better to follow, follow to be better: Towards precise supervision of feature super-resolution for small object detection. InProceedings of the IEEE/CVF international conference on computer vision, pages 9725–9734, 2019

    2019

  15. [23]

    Docci: Descriptions of connected and contrasting images

    Yasumasa Onoe, Sunayana Rane, Zachary Berger, Yonatan Bitton, Jaemin Cho, Roopal Garg, Alexander Ku, Zarana Parekh, Jordi Pont-Tuset, Garrett Tanzer, et al. Docci: Descriptions of connected and contrasting images. InEuropean Conference on Computer Vision, pages 291–309. Springer, 2024

    2024

  16. [24]

    Discriminative fine-tuning of lvlms.arXiv preprint arXiv:2412.04378, 2024

    Yassine Ouali, Adrian Bulat, Alexandros Xenos, Anestis Zaganidis, Ioannis Maniadis Metaxas, Georgios Tzimiropoulos, and Brais Martinez. Discriminative fine-tuning of lvlms.arXiv preprint arXiv:2412.04378, 2024

    2024 arXiv

  17. [25]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. InInternational conference on machine learning, pag...

    2021

  18. [26]

    Dino-x: A unified vision model for open-world object detection and understanding, 2024

    Tianhe Ren, Yihao Chen, Qing Jiang, Zhaoyang Zeng, Yuda Xiong, Wenlong Liu, Zhengyu Ma, Junyi Shen, Yuan Gao, Xiaoke Jiang, Xingyu Chen, Zhuheng Song, Yuhong Zhang, Hongjie Huang, Han Gao, Shilong Liu, Hao Zhang, Feng Li, Kent Yu, and Lei Zhang. Dino-x: A unified vision model ...

    2024

  19. [27]

    Small-object sensitive segmentation using across feature map attention.IEEE transactions on pattern analysis and machine intelligence, 45(5):6289–6306, 2022

    Shengtian Sang, Yuyin Zhou, Md Tauhidul Islam, and Lei Xing. Small-object sensitive segmentation using across feature map attention.IEEE transactions on pattern analysis and machine intelligence, 45(5):6289–6306, 2022

    2022

  20. [28]

    Image retrieval for image- based localization revisited

    Torsten Sattler, Tobias Weyand, Bastian Leibe, and Leif Kobbelt. Image retrieval for image- based localization revisited. InBMVC, page 4, 2012

    2012

  21. [29]

    Eva-clip: Improved training techniques for clip at scale.arXiv preprint arXiv:2303.15389, 2023

    Quan Sun, Yuxin Fang, Ledell Wu, Xinlong Wang, and Yue Cao. Eva-clip: Improved training techniques for clip at scale.arXiv preprint arXiv:2303.15389, 2023

    2023 arXiv

  22. [30]

    Miss detection vs

    Huan Wang, Luping Zhou, and Lei Wang. Miss detection vs. false alarm: Adversarial learning for small object segmentation in infrared images. InProceedings of the IEEE/CVF international conference on computer vision, pages 8509–8518, 2019

    2019

  23. [31]

    Improving text embeddings with large language models

    Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. Improving text embeddings with large language models. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (V olume 1: Long Papers), pages 11897–11916, 2024. 11

    2024

  24. [32]

    Uniir: Training and benchmarking universal multimodal information retrievers

    Cong Wei, Yang Chen, Haonan Chen, Hexiang Hu, Ge Zhang, Jie Fu, Alan Ritter, and Wenhu Chen. Uniir: Training and benchmarking universal multimodal information retrievers. In European Conference on Computer Vision, pages 387–404. Springer, 2024

    2024

  25. [33]

    V*: Guided visual search as a core mechanism in multimodal llms

    Penghao Wu and Saining Xie. V*: Guided visual search as a core mechanism in multimodal llms. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13084–13094, 2024

    2024

  26. [34]

    Described object detection: Liberating object detection with flexible expressions.Advances in Neural Information Processing Systems, 36:79095–79107, 2023

    Chi Xie, Zhao Zhang, Yixuan Wu, Feng Zhu, Rui Zhao, and Shuang Liang. Described object detection: Liberating object detection with flexible expressions.Advances in Neural Information Processing Systems, 36:79095–79107, 2023

    2023

  27. [35]

    Querydet: Cascaded sparse query for accelerating high-resolution small object detection

    Chenhongyi Yang, Zehao Huang, and Naiyan Wang. Querydet: Cascaded sparse query for accelerating high-resolution small object detection. InProceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pages 13668–13677, 2022

    2022

  28. [36]

    From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions

    Peter Young, Alice Lai, Micah Hodosh, and Julia Hockenmaier. From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions. Transactions of the association for computational linguistics, 2:67–78, 2014

    2014

  29. [37]

    Modeling context in referring expressions

    Licheng Yu, Patrick Poirson, Shan Yang, Alexander C Berg, and Tamara L Berg. Modeling context in referring expressions. InComputer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II 14, pages 69–85. Springer, 2016

    2016

  30. [38]

    Sigmoid loss for language image pre-training

    Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. InProceedings of the IEEE/CVF international conference on computer vision, pages 11975–11986, 2023

    2023

  31. [40]

    Long-clip: Unlocking the long-text capability of clip

    Beichen Zhang, Pan Zhang, Xiaoyi Dong, Yuhang Zang, and Jiaqi Wang. Long-clip: Unlocking the long-text capability of clip. InEuropean Conference on Computer Vision, pages 310–325. Springer, 2024

    2024

  32. [41]

    A simple framework for open-vocabulary segmentation and detection

    Hao Zhang, Feng Li, Xueyan Zou, Shilong Liu, Chunyuan Li, Jianwei Yang, and Lei Zhang. A simple framework for open-vocabulary segmentation and detection. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 1020–1031, 2023

    2023

  33. [42]

    Gme: Improving universal multimodal retrieval by multimodal llms.arXiv preprint arXiv:2412.16855, 2024

    Xin Zhang, Yanzhao Zhang, Wen Xie, Mingxin Li, Ziqi Dai, Dingkun Long, Pengjun Xie, Meishan Zhang, Wenjie Li, and Min Zhang. Gme: Improving universal multimodal retrieval by multimodal llms.arXiv preprint arXiv:2412.16855, 2024

    2024 arXiv

  34. [43]

    Vista: Visualized text embedding for universal multi-modal retrieval

    Junjie Zhou, Zheng Liu, Shitao Xiao, Bo Zhao, and Yongping Xiong. Vista: Visualized text embedding for universal multi-modal retrieval. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (V olume 1: Long Papers), pages 3185–3200, 2024

    2024

  35. [44]

    Summarize the [main component/ background/ detail] in the above image in one word:

    Xueyan Zou, Jianwei Yang, Hao Zhang, Feng Li, Linjie Li, Jianfeng Wang, Lijuan Wang, Jianfeng Gao, and Yong Jae Lee. Segment everything everywhere all at once.Advances in neural information processing systems, 36:19769–19782, 2023. 12 Appendix A Environment Complexity of SORCE...

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.