Pith. sign in

REVIEW 3 major objections 5 minor 81 references

Turning a category name into generated images beats text-only retrieval on 15 benchmarks.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Category-level retrieval improves when text queries are converted into multiple generated images, aggregated with a learned attention module, and fused with CLIP text similarity.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection Sound empirical recipe for improving text-only category retrieval with generated visual queries; average gains are real, but 'consistently outperforms' overstates the per-dataset evidence. the 3 major comments →

arxiv 2509.00177 v1 pith:NY24W544 submitted 2025-08-29 cs.CV

Category-level Text-to-Image Retrieval Improved: Bridging the Domain Gap with Diffusion Models and Vision Encoders

Faizan Farooq Khan , Vladan Stojni\'c , Zakaria Laskar , Mohamed Elhoseiny , Giorgos Tolias This is my paper
classification cs.CV
keywords text-to-image retrievalcategory-level retrievalmodality gapdiffusion modelsvision-language modelsDINOv2zero-shot open-vocabulary retrievalsynthetic query generation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether category-level text-to-image retrieval—finding every image of a named class in a large database—can do better than a text-only vision-language model. The authors argue yes, because text and images occupy separated regions of CLIP-style representation space, while image-to-image comparisons with a self-supervised vision encoder are more reliable. Their recipe is to turn the text query into several images with a text-to-image diffusion model, score database images against those generated images using DINOv2, aggregate the generated queries with a small learned attention network, and blend this intra-modal score with the original text-to-image score. Across fifteen benchmarks, the blended score beats the text-only CLIP and SigLIP baselines for class-name queries, for descriptions that never mention the class name, and for the two combined. The load-bearing premise is that generated images capture enough of each class's visual mode in DINOv2 feature space; when a generator misses a key visual cue, such as the hammer-shaped head of a hammerhead shark, retrieval can get worse.

Core claim

Using a text-only VLM as baseline, the paper demonstrates that replacing pure cross-modal search with a hybrid similarity—one term from text-to-image (CLIP/SigLIP) and one from image-to-image (DINOv2) over diffusion-generated queries—consistently improves category retrieval. With class-name queries, the hybrid reaches 67.9 average mAP with CLIP+DINOv2 versus 62.7 for text-only CLIP, and 72.9 with SigLIP+DINOv2 versus 70.3 for text-only SigLIP, averaged over 15 datasets. The improvement also holds when the query is a class description that omits the class name (+6.4 over CLIP, +3.6 over SigLIP) and when description and name are joined. The paper attributes the gain to replacing cross-modal co

What carries the argument

The machinery that carries the argument is the aggregator plus the modality split. The aggregator is a sequence of symmetric self-attention layers: query and key projections are shared, value projections are identity, and the CLS token starts as the average of the generated-image features and is the only token updated across layers. Because the values are identities, the output stays in the same DINOv2 feature space as the database images, so the aggregated query can be compared directly to database images. Around this sits the hybrid score s = (1 − λ) sc + λ si, where sc is VLM text-image similarity, si is DINOv2 image-image similarity between the aggregated generated query and the database

Load-bearing premise

The whole gain rests on generated images being close enough to real category appearance in DINOv2 feature space; when a generator drops a class's defining visual cue, the hybrid score can rank distractors above true images and underperform text-only retrieval.

What would settle it

Take a class whose generated queries are visibly off, such as hammerhead shark, where the generator omits the hammer-shaped head, and compute the per-class difference between the hybrid mAP and the text-only mAP. If removing the misleading generated image or improving only that class's generation fidelity does not change retrieval, or if classes with poor generation fidelity still improve, then the modality-gap explanation fails. A systematic correlation across all ImageNet classes between generation fidelity (measured, for example, by the CLIPScore of the generated image against the class nam

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Any text-only open-vocabulary retrieval pipeline can be upgraded without retraining the VLM or vision encoder; only the small aggregator and λ are trained, on synthetic images.
  • Retrieval quality is bounded by the generator: the paper's upper-bound experiment with real query images shows an average gain of about 11.3 mAP over the text-only baseline, so better diffusion models should translate directly into better retrieval.
  • Class-description-only retrieval becomes practical, letting users find images by appearance or properties when they do not know the category name.
  • Using several complementary generators together (SD, SD-Turbo, FLUX) yields more gains than using one generator, and the method continues to improve as more query images are added.
  • The hybrid is more resistant to image corruption than the text-only baseline, with a smaller drop in mAP as ImageNet-C severity increases.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Because the method works by translating the query into the visual domain rather than by aligning text and image more tightly, I infer the same recipe should transfer to other cross-modal retrieval tasks—audio-to-image, sketch-to-image, or video retrieval—wherever a generative model can produce a query in the target modality.
  • The global scalar λ learned once may leave per-class gains on the table; a query-specific or class-specific λ could adapt to how reliable the generated images are for that class, which the paper's failure cases (e.g., hammerhead shark) suggest would help.
  • I infer that the aggregator could be trained iteratively with the generator: if a diffusion model is fine-tuned to produce queries that maximize the aggregator's retrieval rank, rather than just visual realism, the modality-gap bypass could improve further.
  • The description-only benchmark built with an LLM opens a measurable new task, but its quality depends on the description generator; as LLMs improve, so should this retrieval setting.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes CLETIR, a training-light approach to category-level text-to-image retrieval. Given a text query naming a class, it (1) generates multiple query images with text-to-image diffusion models, (2) encodes them with a vision model (DINOv2) and aggregates the resulting descriptors with a small learned symmetric self-attention network, and (3) fuses the resulting image-to-image similarity with the VLM's text-to-image similarity using a learned scalar lambda. The aggregator and lambda are trained on synthetic images generated from OpenImages class names, with benchmark classes removed. The method is evaluated on 15 datasets under three query types (class name, class description, and both), with CLIP, SigLIP, MetaCLIP, OpenCLIP, and EVA-02-CLIP as VLMs. The main reported result is that on class-name queries the method improves average mAP over text-only CLIP (67.9 vs 62.7) and over text-only SigLIP (72.9 vs 70.3). The paper also reports gains for description-based queries, a small Flickr30k experiment, and robustness results on ImageNet-C.

Significance. If the average improvements are robust, the paper offers a simple and modular way to improve open-vocabulary retrieval by converting text queries into generated visual queries and using a strong vision encoder for intra-modal comparison. A notable strength is that the aggregator and fusion weight are trained on synthetic data with the evaluation benchmarks held out, so the reported retrieval numbers are not fitted. The paper also ships source code and evaluates multiple VLMs and generators, which strengthens reproducibility. The main limitation is that the headline claim of 'consistently outperforming' text-only retrieval is not supported at the per-dataset level, and no uncertainty or per-class analysis is provided. The idea is still valuable if the claim is calibrated to average performance with appropriate statistical support.

major comments (3)
  1. [Section 5.2, Table 1] The abstract and Section 1 state that the approach 'consistently outperforms' retrieval methods that rely only on text queries. Table 1 does not support this at the per-dataset level. For class-name queries, Ours (S,D) is below text-only SigLIP on Stanford Cars (88.2 vs 89.2), Food (92.9 vs 93.1), and FGVC Aircraft (44.1 vs 45.6). Larger drops appear in the description+class rows, e.g., Stanford Cars 85.5 vs 88.5 and FGVC Aircraft 45.3 vs 48.3. The claim is only supported at the level of the 15-dataset average. Please revise the wording to 'on average' or, preferably, augment the evaluation with per-dataset significance tests and a discussion of the exceptions.
  2. [Table 1 and Section 5.2] No variance, confidence intervals, or per-class statistical tests are reported. Several per-dataset differences are small (e.g., Food 92.9 vs 93.1, DTD 50.1 vs 49.9 for CLIP description+class, and Stanford Cars 12.1 vs 12.3 for SigLIP description-only), so the reader cannot tell whether the average improvement is robust or driven by a few large gains. Report bootstrap confidence intervals over classes or datasets, or paired per-class tests, to support the claim that the method improves over the text-only baseline.
  3. [Section 5.5, Table 4, Supplementary Figure 5] The paper's own analysis shows that generator fidelity is a load-bearing factor. Supplementary Figure 5 reports an 11.6% mAP gap between SD-generated image queries and 'perfect' real image queries in the image-only setting, and Table 4 documents classes where the generator misses key cues (e.g., hammerhead shark's hammer-shaped head) and the hybrid score degrades. This does not contradict an average-performance claim, but it directly undermines the word 'consistently' in the abstract. I ask for a per-class or per-dataset success/failure analysis: how often does the hybrid score fall below text-only, and is the drop correlated with generation fidelity or with the number of generated images? This would let the reader assess the practical scope of the method.
minor comments (5)
  1. [Section 4.2, aggregator equation] The definition of the attention layer Al(u) = softmax(phi(u)^T phi(u)) u^T is hard to parse. Please specify the dimensions of u, the softmax axis, and whether the query/key projection is applied per column or per token. The notation u1,i for both input tokens and layer outputs should be disambiguated.
  2. [Section 4.3] The procedure for removing benchmark classes from OpenImages using CLIP nearest neighbors should be described more precisely. Is a threshold used, or is it a nearest-neighbor match? This matters for reproducibility and for assessing potential label leakage through the filtering step.
  3. [Section 4.3, training trick] Fixing the cross-modal similarity of positives to 1 is an ad hoc assumption. Please add an ablation or sensitivity analysis showing the learned lambda values and how the final result changes if this positive-similarity target is varied (e.g., 0.9 or 1.0). This would clarify whether the trick is critical or mostly a stabilizer.
  4. [Supplementary Figure 5] The figure reports results on 14 benchmarks (RESISC45 is excluded). Please state this explicitly in the main text or figure caption, as a reader might otherwise expect 15 datasets.
  5. [Table 4] The column header 'Ground Truth' is slightly misleading; it appears to be a real example image from the class rather than a ground-truth label. Consider renaming it 'Real example' or 'Example database image'.

Circularity Check

0 steps flagged

No significant circularity: aggregator and lambda are trained on synthetic OpenImages classes with benchmark classes removed, and evaluation is held out on real benchmark databases.

full rationale

The paper's derivation chain is self-contained with respect to its empirical claims. The text query is converted into generated image queries by a GDM, then DINOv2 representations are aggregated by a learned attention module and combined with the VLM text similarity via a learned lambda. The aggregator and lambda are trained only on synthetic images generated from OpenImages class names, and the paper explicitly removes classes matching the fifteen test benchmarks ('we remove classes from this corpus that match those of the benchmark datasets'), so no benchmark labels or retrieval results are used to fit the model. The reported evaluation measures retrieval on real benchmark databases, which is a held-out setting. The trick of fixing positive cross-modal similarity to 1 during training affects training dynamics only, not the evaluation labels or the test-time similarity computation. Self-citations to the authors' prior work ([61,62]) appear only in the related-work review as examples of VMs enhancing multimodal tasks; they are not load-bearing and no uniqueness theorem or derivation is imported from them. The paper's own Table 4 and per-dataset drops in Table 1 (e.g., Ours(S,D) below text-only SigLIP on Stanford Cars, Food, and FGVC Aircraft) are correctness/robustness concerns about the 'consistently outperforms' wording, not instances of circular reasoning. No equation is defined in terms of its target, no fitted quantity is renamed as a prediction, and no benchmark result is fed back into the training or model-selection procedure. Accordingly, the appropriate circularity score is 0.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 0 invented entities

The central results depend on pre-trained CLIP, DINOv2, and diffusion checkpoints as unearned inputs, plus a learned aggregator and lambda fitted to synthetic data. The largest unearned input is the assumption that generated images are reliable visual proxies in DINOv2 space; the paper itself documents failure cases. No new physical or conceptual entities are introduced.

free parameters (3)
  • Lambda (modality fusion weight) = learned, not reported
    Trained on synthetic data in Section 4.3; the ablation 'w/o lambda tuning' shows it is worth 4.8 mAP, so the final scalar matters and is not printed in the paper.
  • Aggregator parameters theta = learned, 2 attention layers, not enumerated
    Linear transforms per layer trained by a contrastive loss on 390k synthetic images; the 'average aggregation' ablation shows the module contributes about 6.9 mAP, so the learned weights are central.
  • Contrastive temperature tau = not reported
    Appears in the contrastive loss in Eq. (1); no value or schedule is given, so exact replication depends on the released code.
axioms (5)
  • domain assumption Generated images from a text-to-image model are a sufficient proxy for real images of the same category in DINOv2 feature space.
    Entered in Section 4.1, where visual queries are generated by SD, FLUX, and SD-Turbo; Section 5.5 and Table 4 concede that when the generator misses class-specific cues, the hybrid score drops below text-only.
  • domain assumption DINOv2 descriptors are a better intra-modal ranking signal than CLIP visual descriptors.
    Section 4.2 motivates the choice; Table 1 supports it on average (image-only D 53.7 versus image-only C 41.4), but the superiority is assumed for all 15 datasets and all generated-image distributions.
  • domain assumption Training on synthetic OpenImages classes transfers to unseen benchmark classes after CLIP-based removal of test class names.
    Section 4.3 removes test classes by CLIP nearest neighbor; the completeness of that removal and the synthetic-to-real transfer are unverified beyond the reported test numbers.
  • ad hoc to paper Fixing cross-modal similarity of positives to 1 during training is a valid proxy for perfect text-to-image alignment.
    Section 4.3 states this is an empirical trick used to avoid the trivial lambda equals 0 solution; it is not derived or independently validated.
  • domain assumption Symmetric attention with shared query/key projections and identity value projections preserves compatibility with the database embedding space.
    Section 4.2 justifies the aggregator design; no separate experiment isolates this property from the rest of the method.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Category-level Text-to-Image Retrieval Improved: Bridging the Domain Gap with Diffusion Models and Vision Encoders." pith.science (2026). https://pith.science/paper/NY24W544

@misc{pith2026250900177,
  author       = {Pith},
  title        = {Pith review of: Category-level Text-to-Image Retrieval Improved: Bridging the Domain Gap with Diffusion Models and Vision Encoders},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NY24W544}},
  note         = {Machine review of arXiv:2509.00177}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

This work explores text-to-image retrieval for queries that specify or describe a semantic category. While vision-and-language models (VLMs) like CLIP offer a straightforward open-vocabulary solution, they map text and images to distant regions in the representation space, limiting retrieval performance. To bridge this modality gap, we propose a two-step approach. First, we transform the text query into a visual query using a generative diffusion model. Then, we estimate image-to-image similarity with a vision model. Additionally, we introduce an aggregation network that combines multiple generated images into a single vector representation and fuses similarity scores across both query modalities. Our approach leverages advancements in vision encoders, VLMs, and text-to-image generation models. Extensive evaluations show that it consistently outperforms retrieval methods relying solely on text queries. Source code is available at: https://github.com/faixan-khan/cletir

Figures

Figures reproduced from arXiv: 2509.00177 by Faizan Farooq Khan, Giorgos Tolias, Mohamed Elhoseiny, Vladan Stojni\'c, Zakaria Laskar.

Figure 1
Figure 1. Figure 1: Overview of the proposed method. The text query is used as input to generate [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Image-to-image similarity distributions for synthetic-to-real or synthetic-to-synthetic images. We set the parameter λ to be learnable and ob￾serve that back-propagation needs to be performed only through the intra-modal similarity term. This is due to the fact that the encoder models are frozen. We come up with the following empirical trick, which ef￾fectively increases the performance, and is motivated b… view at source ↗
Figure 3
Figure 3. Figure 3: t-SNE visualization of features for text and image queries from 3 generators, [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Overview of the aggregator network leveraging symmetric self-attention where the [PITH_FULL_IMAGE:figures/full_fig_p017_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Comparison between synthetically generated and real images used as queries. [PITH_FULL_IMAGE:figures/full_fig_p018_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Performance Comparison of Class Name and Class Description Retrieval. Each [PITH_FULL_IMAGE:figures/full_fig_p019_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Comparison of performance drop with increasing corruption severity. The text [PITH_FULL_IMAGE:figures/full_fig_p019_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Performance across all five corruption levels for both text-only baseline and our [PITH_FULL_IMAGE:figures/full_fig_p023_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Class-based retrieval for Caltech101. Bonsai Butterfly Cellphone Crayfish Crocodile Flamingo Platypus Stop sign Tick Water Lily Query Generated Images Ranked Database Images 1 2 3 4 5 6 7 8 9 10 a miniature tree that has been carefully cultivated to thrive in a confined container. the insect with two large wings adorned in vibrant scales is often seen fluttering through gardens. a common device used for co… view at source ↗
Figure 10
Figure 10. Figure 10: Description-based retrieval for Caltech101. [PITH_FULL_IMAGE:figures/full_fig_p024_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Class-based retrieval for Stanford Cars. [PITH_FULL_IMAGE:figures/full_fig_p025_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Description-based retrieval for Stanford Cars. [PITH_FULL_IMAGE:figures/full_fig_p025_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Class-based retrieval for CIFAR-10. Airplane Automobile Bird Cat Deer Dog Frog Horse Ship Truck Query Generated Images Ranked Database Images 1 2 3 4 5 6 7 8 9 10 a typical aircraft is composed of two wings and a central body. a typical vehicle is designed to accommodate four to eight individu￾ls and is supported by four wheels. the creature with wings, a beak, and feathers is known for its ability to fly… view at source ↗
Figure 14
Figure 14. Figure 14: Description-based retrieval for CIFAR-10. [PITH_FULL_IMAGE:figures/full_fig_p026_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Class-based retrieval for CIFAR-100. Beetle Bowl Can Cup Flatfish Man Mouse Oak Tree Telephone Tractor Query Generated Images Ranked Database Images 1 2 3 4 5 6 7 8 9 10 A small, hard-shelled insect with a wide variety of species, known for its adaptability, diverse habitats, and important role in ecosystems as decomposers… A round, concave vessel used for holding, mixing, or serving food, often found in … view at source ↗
Figure 16
Figure 16. Figure 16: Description-based retrieval for CIFAR-100. [PITH_FULL_IMAGE:figures/full_fig_p027_16.png] view at source ↗
Figure 17
Figure 17. Figure 17: Class-based retrieval for DTD. Banded Blotchy Braided Bumpy Gauzy Honeycombed Knitted Lacelike Polka-Dotted Spiraled Query Generated Images Ranked Database Images 1 2 3 4 5 6 7 8 9 10 the material that has been wrapped around something else, usually in a strip or bands, is commonly used for various purposes. the unique pattern of streaks and spots often found in this type of material gives it a distinct a… view at source ↗
Figure 18
Figure 18. Figure 18: Description-based retrieval for DTD [PITH_FULL_IMAGE:figures/full_fig_p028_18.png] view at source ↗
Figure 19
Figure 19. Figure 19: Class-based retrieval for FGVC Aircraft. [PITH_FULL_IMAGE:figures/full_fig_p029_19.png] view at source ↗
Figure 20
Figure 20. Figure 20: Description-based retrieval for FGVC Aircraft. [PITH_FULL_IMAGE:figures/full_fig_p029_20.png] view at source ↗
Figure 21
Figure 21. Figure 21: Class-based retrieval for Flowers 102. Ballon Flower Bishop of Llandaff Camellia Corn Poppy Globe Flower Japanese Anemone Siam Tulip Tiger Lily Tree Mallow Trumpet Creeper Query Generated Images Ranked Database Images 1 2 3 4 5 6 7 8 9 10 the petals of these flowers are usually shaped like bells and there are six of them. the purple petals of the pointed flowers in llandaff are veined with deep red, as no… view at source ↗
Figure 22
Figure 22. Figure 22: Description-based retrieval for Flowers 102. [PITH_FULL_IMAGE:figures/full_fig_p030_22.png] view at source ↗
Figure 23
Figure 23. Figure 23: Class-based retrieval for Food. Beef Carpaccio Beignets Hummus Macarons Omelette Paella Pho Rissoto Sashimi Tuna Tartare Query Generated Images Ranked Database Images 1 2 3 4 5 6 7 8 9 10 a popular dish, thinly sliced raw beef is often accompanied by a flavorful sauce or dressing. fried doughnut-like pastries coated in powdered sugar are a popular treat. it is a thick, creamy paste that is usually light b… view at source ↗
Figure 24
Figure 24. Figure 24: Description-based retrieval for Food [PITH_FULL_IMAGE:figures/full_fig_p031_24.png] view at source ↗
Figure 25
Figure 25. Figure 25: Class-based retrieval for ImageNet. Bath towel Cardigan Castle CD Player Chain Lighthouse Mobile phone Tandem Bicycle Titi Monkey Yellow Garden Spider Query Generated Images Ranked Database Images 1 2 3 4 5 6 7 8 9 10 A soft, absorbent fabric used to dry the body after bathing, often found hanging in bathrooms or folded neatly in linen closets. A versatile knitted garment with an open front, often featuri… view at source ↗
Figure 26
Figure 26. Figure 26: Description-based retrieval for ImageNet. [PITH_FULL_IMAGE:figures/full_fig_p032_26.png] view at source ↗
Figure 27
Figure 27. Figure 27: Class-based retrieval for Kinetics-700. Applying Cream Arm Wrestling Auctioning Bandaging Blasting Sand Blowing Leaves Bouncing on Trampoline Breaking Boards Looking at phone Ski Jumping Query Generated Images Ranked Database Images 1 2 3 4 5 6 7 8 9 10 Gently massaging a soothing, hydrating substance onto the skin, enhancing moisture retention and promoting a smooth, supple texture while offering… A comp… view at source ↗
Figure 28
Figure 28. Figure 28: Description-based retrieval for Kinetics-700. [PITH_FULL_IMAGE:figures/full_fig_p033_28.png] view at source ↗
Figure 29
Figure 29. Figure 29: Class-based retrieval for Oxford Pets. Beagle Birman Bombay Ragdoll Saint Bernard Sphynx Yorkshire Terrier Maine Coon Havanese Pomeranian Query Generated Images Ranked Database Images 1 2 3 4 5 6 7 8 9 10 this breed of dog is known for its small size, short hair, and distinctive droopy ears. this feline breed is known for its medium to long hair, which is soft and silky to the touch. a black cat with brow… view at source ↗
Figure 30
Figure 30. Figure 30: Description-based retrieval for Oxford Pets. [PITH_FULL_IMAGE:figures/full_fig_p034_30.png] view at source ↗
Figure 31
Figure 31. Figure 31: Class-based retrieval for RESISC-45. Airport Basketball Court Bridge Circular Farmland Desert Intersection Mountain Parking Lot Roundabout Wetland Query Generated Images Ranked Database Images 1 2 3 4 5 6 7 8 9 10 An aerial view capturing a sprawling transportation hub, showcasing runways, terminals, and aircraft, providing a detailed perspective of its intricate … An aerial view showcasing a rectangular … view at source ↗
Figure 32
Figure 32. Figure 32: Description-based retrieval for RESISC-45. [PITH_FULL_IMAGE:figures/full_fig_p035_32.png] view at source ↗
Figure 33
Figure 33. Figure 33: Class-based retrieval for SUN397. Abbey Airport Terminal Art gallery Banquet Hall Courtyard Driveway Fountain Gift Shop Patio Thrift shop Query Generated Images Ranked Database Images 1 2 3 4 5 6 7 8 9 10 A serene retreat featuring historic architecture, often set in picturesque landscapes, offering tranquility and a glimpse into monastic life through its ancient … A bustling hub where travelers converge,… view at source ↗
Figure 34
Figure 34. Figure 34: Description-based retrieval for SUN397 [PITH_FULL_IMAGE:figures/full_fig_p036_34.png] view at source ↗
Figure 35
Figure 35. Figure 35: Class-based retrieval for UCF101. Balance Beam Blow dry hair Body weight squats Boxing speed bag Cricket Bowling Cutting in kitchen Haircut Long Jump Playing Guitar Yoyo Query Generated Images Ranked Database Images 1 2 3 4 5 6 7 8 9 10 A narrow, elevated platform where precision and grace are showcased through acrobatic routines, requiring focus and agility to maintain poise and … Transforming damp stran… view at source ↗
Figure 36
Figure 36. Figure 36: Description-based retrieval for UCF101 [PITH_FULL_IMAGE:figures/full_fig_p037_36.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

81 extracted references · 66 canonical work pages · 1 internal anchor

  1. [1]

    Unicom: Universal and compact representation learning for image retrieval

    Xiang An, Jiankang Deng, Kaicheng Yang, Jaiwei Li, Ziyong Feng, Jia Guo, Jing Yang, and Tongliang Liu. Unicom: Universal and compact representation learning for image retrieval. In ICLR, 2023

    2023

  2. [2]

    Shekoofeh Azizi, Simon Kornblith, Chitwan Saharia, Mohammad Norouzi, and David J. Fleet. Synthetic data from diffusion models improves imagenet classification. TMLR, 2023

    work page 2023

  3. [3]

    Food-101 – mining discrim- inative components with random forests

    Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. Food-101 – mining discrim- inative components with random forests. In ECCV, 2014

    work page 2014

  4. [4]

    Coyo-700m: Image-text pair dataset

    Minwoo Byeon, Beomhee Park, Haecheon Kim, Sungjun Lee, Woonhyuk Baek, and Saehoon Kim. Coyo-700m: Image-text pair dataset. https://github.com/ kakaobrain/coyo-dataset, 2022

    work page 2022

  5. [5]

    Emerging properties in self-supervised vision trans- formers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bo- janowski, and Armand Joulin. Emerging properties in self-supervised vision trans- formers. In ICCV, 2021

    work page 2021

  6. [6]

    A short note on the kinetics-700 human action dataset

    Joao Carreira, Eric Noland, Chloe Hillier, and Andrew Zisserman. A short note on the kinetics-700 human action dataset. arXiv preprint arXiv:1907.06987, 2019

    Pith/arXiv arXiv 1907

  7. [7]

    VISOR: towards on-the-fly large-scale object category retrieval

    Ken Chatfield and Andrew Zisserman. VISOR: towards on-the-fly large-scale object category retrieval. In ACCV, 2012

    work page 2012

  8. [8]

    Efficient on-the-fly category retrieval using convnets and gpus

    Ken Chatfield, Karen Simonyan, and Andrew Zisserman. Efficient on-the-fly category retrieval using convnets and gpus. In ACCV, 2014

    work page 2014

  9. [9]

    Parkhi, and Andrew Zisserman

    Ken Chatfield, Relja Arandjelovic, Omkar M. Parkhi, and Andrew Zisserman. On-the- fly learning for visual search of large-scale image and video datasets. International Journal of Multimedia Information Retrieval , 2015

    work page 2015

  10. [10]

    Learning the best pooling strategy for visual semantic embedding

    Jiacheng Chen, Hexiang Hu, Hao Wu, Yuning Jiang, and Changhu Wang. Learning the best pooling strategy for visual semantic embedding. In CVPR, 2021

    work page 2021

  11. [11]

    Bakker, Theodoros Georgiou, Paul W

    Wei Chen, Yu Liu, Weiping Wang, Erwin M. Bakker, Theodoros Georgiou, Paul W. Fieguth, Li Liu, and Michael S. Lew. Deep learning for instance retrieval: A survey. PAMI, 2023

    work page 2023

  12. [12]

    Remote sensing image scene classifica- tion: Benchmark and state of the art

    Gong Cheng, Junwei Han, and Xiaoqiang Lu. Remote sensing image scene classifica- tion: Benchmark and state of the art. Proceedings of the IEEE, 2017. 12 KHAN ET.AL.: CA TEGORY -LEVEL TEXT-TO-IMAGE RETRIEV AL IMPROVED

    work page 2017

  13. [13]

    Repro- ducible scaling laws for contrastive language-image learning

    Mehdi Cherti, Romain Beaumont, Ross Wightman, Mitchell Wortsman, Gabriel Il- harco, Cade Gordon, Christoph Schuhmann, Ludwig Schmidt, and Jenia Jitsev. Repro- ducible scaling laws for contrastive language-image learning. In CVPR, 2023

    work page 2023

  14. [14]

    Probabilistic embeddings for cross-modal retrieval

    Sanghyuk Chun, Seong Joon Oh, Rafael Sampaio De Rezende, Yannis Kalantidis, and Diane Larlus. Probabilistic embeddings for cross-modal retrieval. In CVPR, 2021

    work page 2021

  15. [15]

    Describing textures in the wild

    Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. Describing textures in the wild. In CVPR, 2014

    work page 2014

  16. [16]

    An image is worth 16x16 words: Trans- formers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Trans- formers for image recognition at scale. In ICLR, 2021

    work page 2021

  17. [17]

    Oseledets

    Aleksandr Ermolov, Leyla Mirvakhabova, Valentin Khrulkov, Nicu Sebe, and Ivan V . Oseledets. Hyperbolic vision transformers: Combining improvements in metric learn- ing. In CVPR, 2022

    work page 2022

  18. [18]

    Everingham, L

    M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman. The PAS- CAL Visual Object Classes Challenge 2007 (VOC2007) Results. http://www.pascal- network.org/challenges/VOC/voc2007/workshop/index.html, 2007

    work page 2007

  19. [19]

    Vse++: Improving visual-semantic embeddings with hard negatives

    Fartash Faghri, David J Fleet, Jamie Ryan Kiros, and Sanja Fidler. Vse++: Improving visual-semantic embeddings with hard negatives. In BMVC, 2017

    work page 2017

  20. [20]

    EV A: exploring the limits of masked visual representation learning at scale

    Yuxin Fang, Wen Wang, Binhui Xie, Quan Sun, Ledell Wu, Xinggang Wang, Tiejun Huang, Xinlong Wang, and Yue Cao. EV A: exploring the limits of masked visual representation learning at scale. In CVPR, 2023

    work page 2023

  21. [21]

    Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories

    Li Fei-Fei, Rob Fergus, and Pietro Perona. Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories. In CVPRW, 2004

    work page 2004

  22. [22]

    Pratt, Vivek Ramanujan, Yonatan Bitton, Kalyani Marathe, Stephen Mussmann, Richard Vencu, Mehdi Cherti, Ranjay Krishna, Pang Wei Koh, Olga Saukh, Alexander J

    Samir Yitzhak Gadre, Gabriel Ilharco, Alex Fang, Jonathan Hayase, Georgios Smyrnis, Thao Nguyen, Ryan Marten, Mitchell Wortsman, Dhruba Ghosh, Jieyu Zhang, Eyal Orgad, Rahim Entezari, Giannis Daras, Sarah M. Pratt, Vivek Ramanujan, Yonatan Bitton, Kalyani Marathe, Stephen Mussmann, Richard Vencu, Mehdi Cherti, Ranjay Krishna, Pang Wei Koh, Olga Saukh, Ale...

    work page 2023

  23. [23]

    Beyond instance-level image retrieval: Leveraging captions to learn a global visual representation for semantic retrieval

    Albert Gordo and Diane Larlus. Beyond instance-level image retrieval: Leveraging captions to learn a global visual representation for semantic retrieval. In CVPR, 2017

    work page 2017

  24. [24]

    Benchmarking neural network robustness to common corruptions and perturbations

    Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. ICLR, 2019

    work page 2019

  25. [25]

    Learning semantic concepts and order for image and sentence matching

    Yan Huang, Qi Wu, Chunfeng Song, and Liang Wang. Learning semantic concepts and order for image and sentence matching. In CVPR, 2018. KHAN ET.AL.: CA TEGORY -LEVEL TEXT-TO-IMAGE RETRIEV AL IMPROVED 13

    work page 2018

  26. [26]

    Openclip, July 2021

    Gabriel Ilharco, Mitchell Wortsman, Ross Wightman, Cade Gordon, Nicholas Car- lini, Rohan Taori, Achal Dave, Vaishaal Shankar, Hongseok Namkoong, John Miller, Hannaneh Hajishirzi, Ali Farhadi, and Ludwig Schmidt. Openclip, July 2021. URL https://doi.org/10.5281/zenodo.5143773

    doi:10.5281/zenodo.5143773 2021

  27. [27]

    Retrieval-enhanced contrastive vision-text models

    Ahmet Iscen, Mathilde Caron, Alireza Fathi, and Cordelia Schmid. Retrieval-enhanced contrastive vision-text models. In ICLR, 2024

    work page 2024

  28. [28]

    Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig

    Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc V . Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representation learning with noisy text supervision. In ICML, 2021

    work page 2021

  29. [29]

    In defense of lazy visual grounding for open-vocabulary semantic segmentation

    Dahyun Kang and Minsu Cho. In defense of lazy visual grounding for open-vocabulary semantic segmentation. In ECCV, 2024

    work page 2024

  30. [30]

    3d object representations for fine-grained categorization

    Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 3d object representations for fine-grained categorization. In ICCVW, 2013

    work page 2013

  31. [31]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky. Learning multiple layers of features from tiny images. 2009

    2009

  32. [32]

    The open images dataset v4: Unified image classifica- tion, object detection, and visual relationship detection at scale

    Alina Kuznetsova, Hassan Rom, Neil Alldrin, Jasper Uijlings, Ivan Krasin, Jordi Pont- Tuset, Shahab Kamali, Stefan Popov, Matteo Malloci, Alexander Kolesnikov, Tom Duerig, and Vittorio Ferrari. The open images dataset v4: Unified image classifica- tion, object detection, and visual relationship detection at scale. IJCV, 2020

    work page 2020

  33. [33]

    Flux1 dev

    Black Forest Labs. Flux1 dev. https://github.com/black-forest-labs/ flux, 2024

    work page 2024

  34. [34]

    Proxyclip: Proxy attention improves clip for open-vocabulary segmentation

    Mengcheng Lan, Chaofeng Chen, Yiping Ke, Xinjiang Wang, Litong Feng, and Wayne Zhang. Proxyclip: Proxy attention improves clip for open-vocabulary segmentation. In ECCV, 2024

    work page 2024

  35. [35]

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven C. H. Hoi. BLIP: bootstrapping language-image pre-training for unified vision-language understanding and generation. In ICML, 2022

    work page 2022

  36. [36]

    Visual semantic reason- ing for image-text matching

    Kunpeng Li, Yulun Zhang, Kai Li, Yuanyuan Li, and Yun Fu. Visual semantic reason- ing for image-text matching. In CVPR, 2019

    work page 2019

  37. [37]

    Weixin Liang, Yuhui Zhang, Yongchan Kwon, Serena Yeung, and James Y . Zou. Mind the gap: Understanding the modality gap in multi-modal contrastive representation learning. In NeurIPS, 2022

    work page 2022

  38. [38]

    Learning customized visual models with retrieval-augmented knowledge

    Haotian Liu, Kilho Son, Jianwei Yang, Ce Liu, Jianfeng Gao, Yong Jae Lee, and Chun- yuan Li. Learning customized visual models with retrieval-augmented knowledge. In CVPR, 2023

    work page 2023

  39. [39]

    Mode: Clip data experts via clustering

    Jiawei Ma, Po-Yao Huang, Saining Xie, Shang-Wen Li, Luke Zettlemoyer, Shih-Fu Chang, Wen-Tau Yih, and Hu Xu. Mode: Clip data experts via clustering. In CVPR, 2024

    work page 2024

  40. [40]

    Blaschko, and Andrea Vedaldi

    Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew B. Blaschko, and Andrea Vedaldi. Fine-grained visual classification of aircraft. arXiv preprint arXiv:1306.5151, 2013. 14 KHAN ET.AL.: CA TEGORY -LEVEL TEXT-TO-IMAGE RETRIEV AL IMPROVED

    Pith/arXiv arXiv 2013

  41. [41]

    Dataset diffusion: Diffusion- based synthetic dataset generation for pixel-level semantic segmentation

    Quang Nguyen, Truong Vu, Anh Tran, and Khoi Nguyen. Dataset diffusion: Diffusion- based synthetic dataset generation for pixel-level semantic segmentation. In NeurIPS, 2023

    work page 2023

  42. [42]

    Automated flower classification over a large number of classes

    Maria-Elena Nilsback and Andrew Zisserman. Automated flower classification over a large number of classes. In Indian Conference on Computer Vision, Graphics and Image Processing, 2008

    work page 2008

  43. [43]

    Gpt-3.5-turbo-instruct

    OpenAI. Gpt-3.5-turbo-instruct. https://openai.com/gpt, 2024. Accessed on August 1, 2024

    work page 2024

  44. [44]

    Rabbat, Vasu Sharma, Gabriel Synnaeve, Hu Xu, Hervé Jégou, Julien Mairal, Patrick Labatut, Armand Joulin, and Piotr Bo- janowski

    Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Mahmoud Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po-Yao Huang, Shang-Wen Li, Ishan Misra, Michael G. Rabbat, Vasu Sharma, Gabriel Synnaeve, Hu Xu, Hervé Jégou, Julien Mairal, Patrick...

    work page 2023

  45. [45]

    Frozen transformers in lan- guage models are effective visual encoder layers

    Ziqi Pang, Ziyang Xie, Yunze Man, and Yu-Xiong Wang. Frozen transformers in lan- guage models are effective visual encoder layers. In ICLR, 2024

    work page 2024

  46. [46]

    Parkhi, Andrea Vedaldi, Andrew Zisserman, and C

    Omkar M. Parkhi, Andrea Vedaldi, Andrew Zisserman, and C. V . Jawahar. Cats and dogs. In CVPR, 2012

    2012

  47. [47]

    TIGer: Unifying text-to-image generation and retrieval with large multimodal models

    Leigang Qu, Haochuan Li, Tan Wang, Wenjie Wang, Yongqi Li, Liqiang Nie, and Tat-Seng Chua. TIGer: Unifying text-to-image generation and retrieval with large multimodal models. In ICLR, 2025

    work page 2025

  48. [48]

    Learning transferable visual models from natural lan- guage supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sand- hini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural lan- guage supervision. In ICML, 2021

    work page 2021

  49. [49]

    Zero-shot text-to-image generation

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In ICML, 2021

    2021

  50. [50]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Om- mer. High-resolution image synthesis with latent diffusion models. In CVPR, 2022

    work page 2022

  51. [51]

    Bernstein, Alexander C

    Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael S. Bernstein, Alexander C. Berg, and Li Fei-Fei. Imagenet large scale visual recognition challenge. IJCV, 2015

    work page 2015

  52. [52]

    Den- ton, Seyed Kamyar Seyed Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, Jonathan Ho, David J

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L. Den- ton, Seyed Kamyar Seyed Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, Jonathan Ho, David J. Fleet, and Mohammad Norouzi. Photorealistic text-to-image diffusion models with deep language understanding. In NeurIPS, 2022

    work page 2022

  53. [53]

    Fake it till you make it: Learning transferable representations from synthetic imagenet clones

    Mert Bülent Sariyildiz, Karteek Alahari, Diane Larlus, and Yannis Kalantidis. Fake it till you make it: Learning transferable representations from synthetic imagenet clones. In CVPR, 2023. KHAN ET.AL.: CA TEGORY -LEVEL TEXT-TO-IMAGE RETRIEV AL IMPROVED 15

    work page 2023

  54. [54]

    Adversarial diffusion distillation

    Axel Sauer, Dominik Lorenz, Andreas Blattmann, and Robin Rombach. Adversarial diffusion distillation. arXiv preprint arXiv:2311.17042, 2023

    Pith/arXiv arXiv 2023

  55. [55]

    Hoffmann, Max Argus, V olker Fischer, and Thomas Brox

    Simon Schrodi, David T. Hoffmann, Max Argus, V olker Fischer, and Thomas Brox. Two effects, one trigger: On the modality gap, object bias, and information imbalance in contrastive vision-language representation learning. In ICLR, 2025

    work page 2025

  56. [56]

    LAION-5B: an open large-scale dataset for training next generation image-text models

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wight- man, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Worts- man, Patrick Schramowski, Srivatsa Kundurthy, Katherine Crowson, Ludwig Schmidt, Robert Kaczmarczyk, and Jenia Jitsev. LAION-5B: an open large-scale dataset for training next generation image-text mod...

    work page 2022

  57. [57]

    Kim, Bilge Soran, Raghuraman Krishnamoorthi, Mohamed El- hoseiny, and Vikas Chandra

    Xiaoqian Shen, Yunyang Xiong, Changsheng Zhao, Lemeng Wu, Jun Chen, Chenchen Zhu, Zechun Liu, Fanyi Xiao, Balakrishnan Varadarajan, Florian Bordes, Zhuang Liu, Hu Xu, Hyunwoo J. Kim, Bilge Soran, Raghuraman Krishnamoorthi, Mohamed El- hoseiny, and Vikas Chandra. Longvu: Spatiotemporal adaptive compression for long video-language understanding. arXiv prepr...

    Pith/arXiv arXiv 2024

  58. [58]

    Welle, Mårten Björkman, and Danica Kragic

    Peiyang Shi, Michael C. Welle, Mårten Björkman, and Danica Kragic. Towards under- standing the modality gap in CLIP. In ICLR Workshop on Multimodal Representation Learning: Perks and Pitfalls , 2023

    work page 2023

  59. [59]

    Polysemous visual-semantic embedding for cross-modal retrieval

    Yale Song and Mohammad Soleymani. Polysemous visual-semantic embedding for cross-modal retrieval. In CVPR, 2019

    work page 2019

  60. [60]

    Ucf101: A dataset of 101 human actions classes from videos in the wild

    Khurram Soomro, Amir Roshan Zamir, and Mubarak Shah. Ucf101: A dataset of 101 human actions classes from videos in the wild. arXiv preprint arXiv:1212.0402, 2012

    Pith/arXiv arXiv 2012

  61. [61]

    Label propagation for zero-shot classification with vision-language models

    Vladan Stojni ´c, Yannis Kalantidis, and Giorgos Tolias. Label propagation for zero-shot classification with vision-language models. In CVPR, 2024

    work page 2024

  62. [62]

    Lposs: Label prop- agation over patches and pixels for open-vocabulary semantic segmentation

    Vladan Stojni ´c, Yannis Kalantidis, Jiˇrí Matas, and Giorgos Tolias. Lposs: Label prop- agation over patches and pixels for open-vocabulary semantic segmentation. In CVPR, 2025

    work page 2025

  63. [63]

    EV A-CLIP-18B: scaling CLIP to 18 billion parameters.arXiv preprint arXiv:2402.04252, 2024

    Quan Sun, Jinsheng Wang, Qiying Yu, Yufeng Cui, Fan Zhang, Xiaosong Zhang, and Xinlong Wang. EV A-CLIP-18B: scaling CLIP to 18 billion parameters.arXiv preprint arXiv:2402.04252, 2024

    Pith/arXiv arXiv 2024

  64. [64]

    Cambrian-1: A fully open, vision-centric exploration of multimodal llms

    Peter Tong, Ellis Brown, Penghao Wu, Sanghyun Woo, Adithya Iyer, Sai Charitha Akula, Shusheng Yang, Jihan Yang, Manoj Middepogu, Ziteng Wang, Xichen Pan, Rob Fergus, Yann LeCun, and Saining Xie. Cambrian-1: A fully open, vision-centric exploration of multimodal llms. In NeurIPS, 2024

    work page 2024

  65. [65]

    Eyes wide shut? exploring the visual shortcomings of multimodal llms

    Shengbang Tong, Zhuang Liu, Yuexiang Zhai, Yi Ma, Yann LeCun, and Saining Xie. Eyes wide shut? exploring the visual shortcomings of multimodal llms. In CVPR, 2024

    work page 2024

  66. [66]

    Sus-x: Training-free name- only transfer of vision-language models

    Vishaal Udandarao, Ankush Gupta, and Samuel Albanie. Sus-x: Training-free name- only transfer of vision-language models. In ICCV, 2023. 16 KHAN ET.AL.: CA TEGORY -LEVEL TEXT-TO-IMAGE RETRIEV AL IMPROVED

    work page 2023

  67. [67]

    Set2Model Networks: Learning Discriminatively To Learn Generative Models

    Alexander Vakhitov, Andrey Kuzmin, and Victor S. Lempitsky. Internet-based image retrieval using end-to-end trained deep distributions. arXiv preprint arXiv:1612.07697, 2016

    work page internal anchor Pith review Pith/arXiv arXiv 2016

  68. [68]

    Attention is all you need

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. NeurIPS, 2017

    work page 2017

  69. [69]

    Diffusion feedback helps CLIP see better

    Wenxuan Wang, Quan Sun, Fan Zhang, Yepeng Tang, Jing Liu, and Xinlong Wang. Diffusion feedback helps CLIP see better. In ICLR, 2025

    work page 2025

  70. [70]

    Clip-dinoiser: Teaching CLIP a few DINO tricks

    Monika Wysoczanska, Oriane Siméoni, Michaël Ramamonjisoa, Andrei Bursuc, Tomasz Trzcinski, and Patrick Pérez. Clip-dinoiser: Teaching CLIP a few DINO tricks. In ECCV, 2024

    work page 2024

  71. [71]

    Ehinger, Aude Oliva, and Antonio Torralba

    Jianxiong Xiao, James Hays, Krista A. Ehinger, Aude Oliva, and Antonio Torralba. Sun database: Large-scale scene recognition from abbey to zoo. In CVPR, 2010

    work page 2010

  72. [72]

    De- mystifying CLIP data

    Hu Xu, Saining Xie, Xiaoqing Tan, Po-Yao Huang, Russell Howes, Vasu Sharma, Shang-Wen Li, Gargi Ghosh, Luke Zettlemoyer, and Christoph Feichtenhofer. De- mystifying CLIP data. In ICLR, 2024

    work page 2024

  73. [73]

    Freemask: Synthetic images with dense annotations make stronger segmentation mod- els

    Lihe Yang, Xiaogang Xu, Bingyi Kang, Yinghuan Shi, and Hengshuang Zhao. Freemask: Synthetic images with dense annotations make stronger segmentation mod- els. In NeurIPS, 2023

    work page 2023

  74. [74]

    From image descrip- tions to visual denotations: New similarity metrics for semantic inference over event descriptions

    Peter Young, Alice Lai, Micah Hodosh, and Julia Hockenmaier. From image descrip- tions to visual denotations: New similarity metrics for semantic inference over event descriptions. Transactions of the Association for Computational Linguistics , 2014

    work page 2014

  75. [75]

    Coca: Contrastive captioners are image-text foundation models

    Jiahui Yu, Zirui Wang, Vijay Vasudevan, Legg Yeung, Mojtaba Seyedhosseini, and Yonghui Wu. Coca: Contrastive captioners are image-text foundation models. TMLR, 2022

    2022

  76. [76]

    Florence: A new foundation model for computer vision

    Lu Yuan, Dongdong Chen, Yi-Ling Chen, Noel Codella, Xiyang Dai, Jianfeng Gao, Houdong Hu, Xuedong Huang, Boxin Li, Chunyuan Li, Ce Liu, Mengchen Liu, Zicheng Liu, Yumao Lu, Yu Shi, Lijuan Wang, Jianfeng Wang, Bin Xiao, Zhen Xiao, Jianwei Yang, Michael Zeng, Luowei Zhou, and Pengchuan Zhang. Florence: A new foundation model for computer vision. arXiv prepr...

    Pith/arXiv arXiv 2021

  77. [77]

    Sigmoid loss for language image pre-training

    Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In ICCV, 2023

    2023

  78. [78]

    Tip-adapter: Training-free clip-adapter for better vision- language modeling

    Renrui Zhang, Rongyao Fang, Peng Gao, Wei Zhang, Kunchang Li, Jifeng Dai, Yu Qiao, and Hongsheng Li. Tip-adapter: Training-free clip-adapter for better vision- language modeling. arXiv preprint arXiv:2111.03930, 2021

    Pith/arXiv arXiv 2021

  79. [79]

    Prompt, generate, then cache: Cascade of foundation models makes strong few-shot learners

    Renrui Zhang, Xiangfei Hu, Bohao Li, Siyuan Huang, Hanqiu Deng, Yu Qiao, Peng Gao, and Hongsheng Li. Prompt, generate, then cache: Cascade of foundation models makes strong few-shot learners. In CVPR, 2023

    work page 2023

  80. [80]

    Places: A 10 million image database for scene recognition

    Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 million image database for scene recognition. PAMI, 2017

    work page 2017

Showing first 80 references.

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.