Pith. sign in

REVIEW 4 major objections 8 minor 66 references

Visual Textualization for Image Prompted Object Detection

T0 review · 4 major / 8 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that projecting a few visual exemplars into the text feature space of a frozen object-level vision-language model lets it detect novel and rare categories that text prompts alone miss, without fine-tuning the model or…

desk verdict Strong few-shot detection results with a genuinely new mechanism, but the 'textualization' claim is under-verified; still worth refereeing. read the letter →

arxiv 2506.23785 v1 pith:ERZG36Q6 submitted 2025-06-30 cs.CV

Yongjian Wu , Yang Zhou , Jiya Saiyin , Bingzheng Wei , Yan Xu This is my paper
classification cs.CV
keywords visualtextualizationimagepromptingfew-shotobjectdetectionobject-levelvision-languagemodelsopen-settextfeaturespaceprojectionmulti-scaletextualizingblockzero-shottransfer
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces VisTex-OVLM, an image-prompted object detection method built on the idea that a few support images can be converted into tokens inside an object-level vision-language model's (OVLM's) text feature space and concatenated with the text prompt. The central claim is that this 'visual textualization' injects the visual semantics that text prompts lack, so a frozen pre-trained OVLM such as GLIP or GroundingDINO can detect categories that were rare or absent during pre-training, without any modification to the model's weights or architecture. The paper further claims that because the OVLM is untouched, its pre-trained object-text alignment is preserved, which is what prior fine-tuning and image-prompting methods damage. A sympathetic reader should care because, if this holds, few-shot object detection becomes a prompt-composition problem: train one lightweight projector on base classes, then at inference project any novel-class exemplar into a textualized visual token. The paper reports state-of-the-art results on the PASCAL VOC and MSCOCO few-shot benchmarks and strong transfer to LVIS, low-mAP ODinW35 subsets, and five medical datasets with minimal overlap with pre-training data.

What carries the argument

The central mechanism is the textualized visual token. The paper defines visual textualization as projecting support-image visual features into the OVLM's text feature space, specifically into the BERT token space that the frozen text encoder produces. The multi-scale textualizing block (MSTB) is a parameter-shared stack of 3×3 strided convolutions followed by an MLP that downsamples each scale of the visual encoder's intermediate features and maps them to the text dimension $d_T$; the multi-stage fusion (MSF) then max-pools these per-stage tokens into one token per support image. The work it does is to convert an image prompt into the exact token format the OVLM already consumes, so the model's original architecture and alignment are untouched and the only trainable parameters are the MSTB's. The design choices that carry the argument are shared parameters across scales, concatenation of $K$-shot tokens rather than fusion (preserving each shot's information), and max pooling across stages as the non-parametric fusion.

What would settle it

Measure the Object365 object-text cosine similarity distribution after inserting textualized visual tokens into a frozen GLIP: if the distribution shifts as much as under full fine-tuning, the alignment-preservation claim would collapse. As a second check, evaluate MSTB on support classes that have no overlap with OVLM pre-training; if the gain over zero-shot vanishes, the method is re-weighting pre-existing object-text knowledge rather than introducing genuinely new visual semantics.

Watch

Extended reading notes

Core claim

On its own terms, the paper's discovery is that an OVLM's frozen cross-attention text branch can be driven directly by visual information if that information is first projected into the model's text feature space. A compact multi-scale textualizing block (MSTB) with shared parameters maps the multi-scale visual features of a prompt-engineered support image (background blurred around the target box) into a single textualized visual token per stage; a non-parametric multi-stage fusion max-pools these tokens so each support image contributes one token. At inference the tokens are concatenated with BERT text tokens and fed into the unmodified pre-trained OVLM. The paper argues this preserves the model's object-text alignment because nothing in the OVLM is modified; only the lightweight projector is trained on base classes. Empirically the paper shows that any weight or structural modification, including the cross-attention modules added by prior image-prompting work, shifts the Object365 object-text similarity distribution, whereas visual textualization leaves it near the pre-trained distribution. The central asserted result is that this alignment preservation transfers to open-set detection: VisTex-GLIP and VisTex-DINO outperform baselines on LVIS MiniVal, on 11 low-mAP ODinW35 subsets, and on five medical datasets, and VisTex-GLIP sets the best results on the standard PASCAL VOC and MSCOCO few-shot benchmarks.

Load-bearing premise

The load-bearing premise is that a frozen OVLM's text feature space will accept projected visual tokens as meaningful semantic inputs, so concatenating them with BERT text tokens and passing the mixture through frozen cross-attention guides detection rather than distorting it; this is validated only empirically in the paper.

Editorial extensions

If this is right

  • Few-shot detection of novel classes can be achieved by training only the lightweight MSTB projector on base classes, leaving the OVLM's weights frozen.
  • Because the detector itself is unchanged, base-class performance is retained, meeting generalized few-shot detection requirements without knowledge forgetting.
  • The same recipe transfers across OVLMs: the paper demonstrates working instances on GLIP, GroundingDINO, RegionCLIP, and FIBER.
  • Image prompts complement text prompts rather than replace them; ablations show that dropping the text prompt degrades attention and performance, so text remains the semantic anchor.
  • Open-set transfer improves on categories with minimal pre-training overlap, on both natural and medical imaging domains.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If textualized visual tokens are genuinely read as semantic tokens by frozen cross-attention, then the recipe should work on other OVLMs and possibly on image-level VLMs with cross-attention text branches; applying MSTB to such a model is a direct testable extension not reported in the paper.
  • The method's success suggests the OVLM text space operates as a shared semantic substrate rather than a purely linguistic encoding; probing whether textualized tokens activate the same attention patterns as class-name tokens would test this interpretation.
  • The paper reports failures on dense and small objects, attributed to weak small-object representations in the pre-trained OVLM and the difficulty of fitting novel distributions from few samples; better support-image selection or scale-aware tokenization is a plausible extension the paper leaves open.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 8 minor

Summary. The paper proposes VisTex-OVLM, an image-prompted object detection method that projects support images into the text feature space of a frozen object-level vision-language model (OVLM) via multi-scale textualizing blocks (MSTB) and a non-parametric multi-stage fusion (MSF) strategy. The resulting 'textualized visual tokens' are concatenated with BERT text tokens and fed into unmodified GLIP or GroundingDINO. The paper reports state-of-the-art few-shot results on PASCAL VOC and MSCOCO, strong open-set results on LVIS and 16 datasets with minimal overlap with pre-training data, and additional compatibility experiments on RegionCLIP and FIBER. The central claim is that this design preserves the OVLM's pre-trained object-text alignment while introducing new visual semantics.

Significance. If the mechanism is as claimed, this is a practically valuable approach: it enables few-shot and open-set detection without fine-tuning the OVLM, adds only a small trainable module, and is demonstrated across multiple OVLMs and domains, including medical imaging. The empirical gains on standard FSOD benchmarks are substantial (e.g., Table 2, VisTex-GLIP mean AP50 71.8 vs 71.2 for MTL-FSOD; Table 3, VisTex-DINO outperforms GroundingDINO-FF at several shot settings). The paper also provides useful ablations on multi-scale textualization, stage fusion, shot fusion, and image prompt engineering. However, the central premise—that projected visual tokens are semantically in-distribution for the frozen text encoder—is asserted rather than demonstrated, and the alignment-preservation claim is supported only by an informal figure with no quantitative comparison. These gaps are load-bearing for the claimed novelty.

major comments (4)
  1. [Sec. 3.3.1, Eqs. (3)-(5)] The central claim of 'visual textualization' is that MSTB maps visual features into the text feature space so that the concatenated token fPS is semantically meaningful to the frozen BERT/cross-attention stack. The paper provides no evidence for this: there is no analysis of the norms, cosine similarities to word/class embeddings, or nearest-neighbor statistics of fPS, and no term in the training loss enforces distribution matching with BERT token embeddings. Without such evidence, the detection gains could be explained by fPS acting as an arbitrary side-channel into cross-attention rather than as a textual token, which would undercut the claimed distinction from MQ-Det. Please add a direct distributional analysis (e.g., compare fPS to BERT token embeddings on base classes) and an ablation that replaces fPS with random or non-textual projected vectors to show that textualization is necessary.
  2. [Introduction, Fig. 1; Sec. 3.3.2] The claim that VisTex-OVLM 'preserves OVLM's pre-trained object-text alignment' is not quantitatively measured. Figure 1 is invoked for method 6, but the text reports no numerical curve or metric for VisTex-GLIP, and the figure itself is not included in the submitted manuscript. Freezing the OVLM weights does not guarantee unchanged behavior once an extra token is concatenated to the text sequence. Please report a quantitative alignment metric on a source domain (e.g., mean/quantiles of Object365 object-text cosine similarity before and after applying VisTex) and compare it against GLIP-ZS, GLIP-FF, and MQ-Det.
  3. [Sec. 4.3, Tables 1-3] The paper states that open-set results are 'the average of five experimental runs' but reports no standard deviations, confidence intervals, or significance tests. Several improvements over baselines are small (e.g., Table 1, VisTex-GLIP vs GLIP-FF on MoNu and ConSeP; Table 3, VisTex-DINO vs GroundingDINO-FF at 1-2 shots). Without variance information, the SOTA claims on these benchmarks are not yet established. Please include error bars or a table of standard deviations, and flag differences that are not significant.
  4. [Sec. 3.3.2, Eq. (6)] For K-shot multi-class prompts, P0 has |C|·(N+K) tokens. The authors do not discuss whether these tokens exceed BERT's maximum sequence length, how positional encodings are handled for the concatenated textualized tokens (which are inserted after BERT(t) and therefore may lack positional embeddings), or whether the positional invariance assumption was validated. This is a concrete manifestation of the in-distribution concern; please clarify how positional encodings and sequence-length limits are handled and verify that performance is not sensitive to token position or sequence-length truncation.
minor comments (8)
  1. [Sec. 3.3.1, Eq. (4)] Equation (4) uses fRi S in the multi-stage fusion operation, but the surrounding text states that MSF integrates textualized visual features fP i S; the notation should be corrected to fP i S.
  2. [Sec. 3.3.1, Eq. (1)] The product notation in Eq. (1), Q_{j=0}^{M-2} Convi,(j)_down, is difficult to parse; please define the order of application and the meaning of the product operator explicitly.
  3. [Table 5 and Sec. 4.5] The sentence ''×' in the 'MSF' column for 'VisTex-GLIP' indicates that textualization is applied only in stage 1' is ambiguous because several rows have × in that column. Please specify that it refers to the row with MSTB sharing enabled and MSF disabled.
  4. [Acknowledgements and Tables 2-3] There are several typos: 'Bejing' should be 'Beijing', 'NeurlPS' should be 'NeurIPS', and 'Luddecke' should be 'Lüddecke'.
  5. [Fig. 1] Figure 1 is referenced in the introduction but does not appear in the submitted manuscript; please ensure the figure is included and legible, with method labels and quantitative legend values.
  6. [Sec. 4.3, Tables 1-3] The statement that results are the average of five experimental runs appears only in Sec. 4.3; please also indicate it in the table captions and state whether the same averaging applies to the FSOD tables.
  7. [Supplementary Sec. 10.4] The text says 'Tab. 12 in the main text shows multi-stage fusion's effectiveness,' but Table 12 is in the supplementary material; please correct the cross-reference.
  8. [Abstract and Sec. 4.1] The abstract claims the open-set datasets have 'minimal overlap' with OVLM pre-training data, but this is not quantified for LVIS, which shares categories with Object365/COCO; please clarify which datasets this claim applies to and how overlap was measured.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: few-shot and open-set results are evaluated on held-out novel classes with externally trained OVLM weights.

full rationale

The paper's claimed derivation chain is not circular. MSTB is the only trainable component and is trained on base-set classes with OVLM's pre-training losses; all few-shot (PASCAL VOC, MSCOCO) and open-set (LVIS, ODinW, medical) evaluations use novel classes and support images unseen during MSTB training, so the reported AP values are genuine held-out generalization measurements rather than quantities forced by construction. No fitted parameter is renamed as a prediction: the textualized visual token fPS is concatenated into the frozen OVLM text prompt, but its effectiveness is judged only on disjoint query categories. The paper does not rely on a load-bearing self-citation chain; prior-work citations such as GLIP, MQ-Det, and CLIPSeg are external, and the method is instantiated on public OVLM checkpoints and compared against external baselines. One caveat is that the central premise (projected visual tokens are semantically meaningful in OVLM's text space) and the alignment-preservation claim are supported mainly through downstream detection gains, with the text referring to Fig. 1 method 6 without reporting a quantitative alignment curve for VisTex-GLIP in the main text; this is a missing-evidence or correctness concern, not a circular reduction, because the downstream detection results are not used as the definition or fit of the alignment claim. Under the hard rules, no quoted step can be exhibited where an equation or fitted value reduces to its own input, so the appropriate finding is no significant circularity.

Assumptions & free parameters 6 free parameters · 4 assumptions · 1 invented entities

The central claim rests on two main assumptions: that projected visual tokens retain enough semantic information to be useful when concatenated with text tokens, and that the projector trained on base classes transfers to novel categories. The design choices (stage range, fusion mode, prompt engineering) are not derived from theory but are validated by ablations. A modest set of hand-chosen hyperparameters and a learned projector constitute the main free parameters.

free parameters (6)
  • MSTB trainable weights = 63.06M parameters (VisTex-GLIP full config)
    Learned projection weights from visual to text feature space; trained on MSCOCO base set, not manually chosen.
  • Number of visual encoder stages fused = 8 stages (1 to 8)
    Ablation shows 1 to 8 best; choice affects performance.
  • Multi-scale feature count M = 5 for GLIP-L, 3 for GroundingDINO-T
    Determined by backbone feature pyramid, not tuned.
  • Stage fusion mode = max pooling
    Ablation shows max pooling best among non-parametric options.
  • Image prompt engineering (BG blur) = background blur with intensity 0.1, Gaussian kernel size 15, standard deviation 3
    Heuristic preprocessing that ablations show works best.
  • Input spatial size H, W = 100 by 100 for GLIP-L
    Set based on GLIP-L feature sizes; W adjusted for GroundingDINO via bilinear interpolation.
assumptions (4)
  • domain assumption The OVLM's text feature space can serve as a common semantic receptacle for projected visual features; visual tokens concatenated to BERT tokens will be processed meaningfully by frozen cross-attention.
    Core to visual textualization; no theoretical guarantee, only empirical validation. Invoked in Sec. 3.3.1, Eq. 5.
  • domain assumption Training MSTB on base classes generalizes to novel classes absent from training.
    Standard few-shot assumption; supported by results but not guaranteed. Invoked in Sec. 3.3.2.
  • ad hoc to paper Non-parametric max pooling across stages is sufficient fusion.
    Chosen after ablation; no principled derivation. Invoked in Sec. 3.3.1, Eq. 4 and Sec. 10.4.
  • ad hoc to paper Background blur (image prompt engineering) preserves object information while removing distractors.
    Heuristic design choice borrowed from CLIPSeg; validated by ablation in Sec. 10.5.
invented entities (1)
  • Textualized visual tokens independent evidence
    purpose: Represent support images as tokens in the text feature space to prompt a frozen OVLM.
    Not a physical entity; a new representation. Its effectiveness is tested on external benchmarks in Sec. 4.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Visual Textualization for Image Prompted Object Detection." pith.science (2026). https://pith.science/paper/ERZG36Q6

@misc{pith2026250623785,
  author       = {Pith},
  title        = {Pith review of: Visual Textualization for Image Prompted Object Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ERZG36Q6}},
  note         = {Machine review of arXiv:2506.23785}
}
read the original abstract

We propose VisTex-OVLM, a novel image prompted object detection method that introduces visual textualization -- a process that projects a few visual exemplars into the text feature space to enhance Object-level Vision-Language Models' (OVLMs) capability in detecting rare categories that are difficult to describe textually and nearly absent from their pre-training data, while preserving their pre-trained object-text alignment. Specifically, VisTex-OVLM leverages multi-scale textualizing blocks and a multi-stage fusion strategy to integrate visual information from visual exemplars, generating textualized visual tokens that effectively guide OVLMs alongside text prompts. Unlike previous methods, our method maintains the original architecture of OVLM, maintaining its generalization capabilities while enhancing performance in few-shot settings. VisTex-OVLM demonstrates superior performance across open-set datasets which have minimal overlap with OVLM's pre-training data and achieves state-of-the-art results on few-shot benchmarks PASCAL VOC and MSCOCO. The code will be released at https://github.com/WitGotFlg/VisTex-OVLM.

Figures

Figures reproduced from arXiv: 2506.23785 by the authors.

Figure 1
Figure 1. Frequency distribution of feature cosine similarity be￾tween object-text pairs from Object365 dataset after transferring the Object365-pretrained GLIP to MSCOCO with different methods. modules within the text encoder, enabling exemplar images to modulate the text prompts token-wisely and introduce the visual information of novel classes during inference. However, MQ-Det only re-weights tokens of existing text prompt… view at source ↗
Figure 2
Figure 2. Overview of VisTex-OVLM. VisTex-OVLM performs visual textualization of support images through parameter-shared multi-scale [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Comparison output visualizations on COCO. (a) Meta [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Attention heatmaps comparison. (a) Ground truth, (b) [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Visualization of VisTex-GLIP’s 10-shot object detection [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]
Figure 6
Figure 6. Figure 6: Output visualizations for medical datasets. The first row [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: Failure cases. (a) GLIP-ZS, (b) GLIP-FF, (d) GLIP [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 8
Figure 8. Figure 8: Comparison output visualizations on ODinW13. From top row to the bottom: Aquarium, AerialDrone, Rabbits, EgoHands, [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

66 extracted references · 53 canonical work pages

  1. [1]

    Lung image database consor- tium: developing a resource for the medical imaging research community

    Samuel G Armato III, Geoffrey McLennan, Michael F McNitt-Gray, Charles R Meyer, David Yankelevitz, Denise R Aberle, Claudia I Henschke, Eric A Hoffman, Ella A Kaze- rooni, Heber MacMahon, et al. Lung image database consor- tium: developing a resource for the medical imaging research community. Radiology, 232(3):739–748, 2004. 5

    work page 2004

  2. [2]

    Exploring visual prompts for adapting large- scale models

    Hyojin Bahng, Ali Jahanian, Swami Sankaranarayanan, and Phillip Isola. Exploring visual prompts for adapting large- scale models. arXiv preprint arXiv:2203.17274, 2022. 3

    arXiv 2022

  3. [3]

    Fs-detr: Few-shot detection transformer with prompting and without re-training

    Adrian Bulat, Ricardo Guerrero, Brais Martinez, and Geor- gios Tzimiropoulos. Fs-detr: Few-shot detection transformer with prompting and without re-training. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 11793–11802, 2023. 1

    work page 2023

  4. [4]

    Apollo: Unified adapter and prompt learning for vision language models

    Sanjoy Chowdhury, Sayan Nag, and Dinesh Manocha. Apollo: Unified adapter and prompt learning for vision language models. In Proceedings of the 2023 Conference on Empiri- cal Methods in Natural Language Processing, pages 10173– 10187, 2023. 3

    work page 2023

  5. [5]

    Coarse-to-fine vision-language pre-training with fusion in the backbone.NeurIPS, 35:32942–32956, 2022

    Zi-Yi Dou et al. Coarse-to-fine vision-language pre-training with fusion in the backbone.NeurIPS, 35:32942–32956, 2022. 1, 3, 4

    work page 2022

  6. [6]

    s- adaptive decoupled prototype for few-shot object detection

    Jinhao Du, Shan Zhang, Qiang Chen, Haifeng Le, Yanpeng Sun, Yao Ni, Jian Wang, Bin He, and Jingdong Wang. s- adaptive decoupled prototype for few-shot object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 18950–18960, 2023. 6, 7

    work page 2023

  7. [7]

    Learning to prompt for open-vocabulary object detection with vision-language model

    Yu Du, Fangyun Wei, Zihe Zhang, Miaojing Shi, Yue Gao, and Guoqi Li. Learning to prompt for open-vocabulary object detection with vision-language model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14084–14093, 2022. 1

    work page 2022

  8. [8]

    The turking test: Can lan- guage models understand instructions? arXiv preprint arXiv:2010.11982, 2020

    Avia Efrat and Omer Levy. The turking test: Can lan- guage models understand instructions? arXiv preprint arXiv:2010.11982, 2020. 3

    arXiv 2010

Show all 66 references
  1. [9]

    The pascal visual object classes (voc) challenge

    Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. The pascal visual object classes (voc) challenge. International journal of computer vision, 88:303–338, 2010. 2, 6

    2010

  2. [10]

    Few- shot object detection with attention-rpn and multi-relation detector

    Qi Fan, Wei Zhuo, Chi-Keung Tang, and Yu-Wing Tai. Few- shot object detection with attention-rpn and multi-relation detector. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4013–4022,

  3. [11]

    Nuclei grading of clear cell renal cell carcinoma in histopatho- logical image by composite high-resolution network

    Zeyu Gao, Jiangbo Shi, Xianli Zhang, Yang Li, Haichuan Zhang, Jialun Wu, Chunbao Wang, Deyu Meng, and Chen Li. Nuclei grading of clear cell renal cell carcinoma in histopatho- logical image by composite high-resolution network. In In- ternational Conference on Medical Image Co...

  4. [12]

    Hover-net: Simultaneous segmentation and classification of nuclei in multi-tissue histology images

    Simon Graham, Quoc Dang Vu, Shan E Ahmed Raza, Ayesha Azam, Yee Wah Tsang, Jin Tae Kwak, and Nasir Rajpoot. Hover-net: Simultaneous segmentation and classification of nuclei in multi-tissue histology images. Medical Image Anal- ysis, 58:101563, 2019. 5

    2019

  5. [13]

    A systematic survey of prompt engineer- ing on vision-language foundation models

    Jindong Gu, Zhen Han, Shuo Chen, Ahmad Beirami, Bailan He, Gengyuan Zhang, Ruotong Liao, Yao Qin, V olker Tresp, and Philip Torr. A systematic survey of prompt engineer- ing on vision-language foundation models. arXiv preprint arXiv:2307.12980, 2023. 3

    2023 arXiv

  6. [14]

    Open- vocabulary object detection via vision and language knowl- edge distillation

    Xiuye Gu, Tsung-Yi Lin, Weicheng Kuo, and Yin Cui. Open- vocabulary object detection via vision and language knowl- edge distillation. arXiv preprint arXiv:2104.13921 , 2021. 3

    2021 arXiv

  7. [15]

    Dp-ddcl: A discriminative prototype with dual decou- pled contrast learning method for few-shot object detection

    Yinsai Guo, Liyan Ma, Xiangfeng Luo, and Shaorong Xie. Dp-ddcl: A discriminative prototype with dual decou- pled contrast learning method for few-shot object detection. Knowledge-Based Systems, 297:111964, 2024. 1, 3, 6, 7

    2024

  8. [16]

    Lvis: A dataset for large vocabulary instance segmentation

    Agrim Gupta, Piotr Dollar, and Ross Girshick. Lvis: A dataset for large vocabulary instance segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5356–5364, 2019. 2, 5

    2019

  9. [17]

    Few-shot object detection with foundation models

    Guangxing Han and Ser-Nam Lim. Few-shot object detection with foundation models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 28608–28618, 2024. 1, 3, 6, 7

    2024

  10. [18]

    Query adaptive few-shot object detec- tion with heterogeneous graph convolutional networks

    Guangxing Han, Yicheng He, Shiyuan Huang, Jiawei Ma, and Shih-Fu Chang. Query adaptive few-shot object detec- tion with heterogeneous graph convolutional networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3263–3272, 2021. 2

    2021

  11. [19]

    Multi-modal few- shot object detection with meta-learning-based cross-modal prompting

    Guangxing Han, Long Chen, Jiawei Ma, Shiyuan Huang, Rama Chellappa, and Shih-Fu Chang. Multi-modal few- shot object detection with meta-learning-based cross-modal prompting. arXiv preprint arXiv:2204.07841, 2022. 1, 3

    2022 arXiv

  12. [20]

    Meta faster r-cnn: Towards accurate few-shot object detection with attentive feature alignment

    Guangxing Han, Shiyuan Huang, Jiawei Ma, Yicheng He, and Shih-Fu Chang. Meta faster r-cnn: Towards accurate few-shot object detection with attentive feature alignment. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 780–789, 2022. 2

    2022

  13. [22]

    Few-shot object detection with fully cross- transformer

    Guangxing Han, Jiawei Ma, Shiyuan Huang, Long Chen, and Shih-Fu Chang. Few-shot object detection with fully cross- transformer. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5321–5330,

  14. [23]

    Few-shot object detection via variational feature aggregation

    Jiaming Han, Yuqiang Ren, Jian Ding, Ke Yan, and Gui- Song Xia. Few-shot object detection via variational feature aggregation. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 755–763, 2023. 6

    2023

  15. [24]

    Visual prompt tuning

    Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, and Ser-Nam Lim. Visual prompt tuning. In European Conference on Computer Vision, pages 709–727. Springer, 2022. 3

    2022

  16. [25]

    Bert: Pre-training of deep bidirectional transform- ers for language understanding

    Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. Bert: Pre-training of deep bidirectional transform- ers for language understanding. InProceedings of naacL-HLT, page 2. Minneapolis, Minnesota, 2019. 4

    2019

  17. [26]

    Maple: Multi- modal prompt learning

    Muhammad Uzair Khattak, Hanoona Rasheed, Muhammad Maaz, Salman Khan, and Fahad Shahbaz Khan. Maple: Multi- modal prompt learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 19113–19122, 2023. 3, 8

    2023

  18. [27]

    A dataset and a technique for generalized nuclear segmentation for computa- tional pathology

    Neeraj Kumar, Ruchika Verma, Sanuj Sharma, Surabhi Bhar- gava, Abhishek Vahadane, and Amit Sethi. A dataset and a technique for generalized nuclear segmentation for computa- tional pathology. IEEE transactions on medical imaging, 36 (7):1550–1560, 2017. 5

    2017

  19. [28]

    F-vlm: Open-vocabulary object detection upon frozen vision and language models

    Weicheng Kuo, Yin Cui, Xiuye Gu, AJ Piergiovanni, and Anelia Angelova. F-vlm: Open-vocabulary object detection upon frozen vision and language models. arXiv preprint arXiv:2209.15639, 2022. 3

    2022 arXiv

  20. [29]

    Elevater: A benchmark and toolkit for evaluating language-augmented visual models

    Chunyuan Li, Haotian Liu, Liunian Li, Pengchuan Zhang, Jyoti Aneja, Jianwei Yang, Ping Jin, Houdong Hu, Zicheng Liu, Yong Jae Lee, et al. Elevater: A benchmark and toolkit for evaluating language-augmented visual models. Advances in Neural Information Processing Systems , 35:9...

  21. [30]

    Disentangle and remerge: interventional knowledge distillation for few-shot object detection from a conditional causal perspective

    Jiangmeng Li, Yanan Zhang, Wenwen Qiang, Lingyu Si, Chengbo Jiao, Xiaohui Hu, Changwen Zheng, and Fuchun Sun. Disentangle and remerge: interventional knowledge distillation for few-shot object detection from a conditional causal perspective. In Proceedings of the AAAI Conferen...

    2023

  22. [31]

    Grounded language- image pre-training

    Liunian Harold Li, Pengchuan Zhang, Haotian Zhang, Jian- wei Yang, Chunyuan Li, Yiwu Zhong, Lijuan Wang, Lu Yuan, Lei Zhang, Jenq-Neng Hwang, et al. Grounded language- image pre-training. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, p...

    2022

  23. [32]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings...

    2014

  24. [33]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection

    Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. arXiv preprint arXiv:2303.05499, 2023. 1, 2, 3, 4, 6

    2023 arXiv

  25. [34]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10012–10022, 2021. 4

    2021

  26. [35]

    Breaking immutable: Information-coupled prototype elaboration for few-shot ob- ject detection

    Xiaonan Lu, Wenhui Diao, Yongqiang Mao, Junxi Li, Pei- jin Wang, Xian Sun, and Kun Fu. Breaking immutable: Information-coupled prototype elaboration for few-shot ob- ject detection. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 1844–1852, 2023. 6

    2023

  27. [36]

    Image segmentation us- ing text and image prompts

    Timo Lüddecke and Alexander Ecker. Image segmentation us- ing text and image prompts. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 7086–7096, 2022. 1, 3, 4, 5, 2

    2022

  28. [37]

    Digeo: Discriminative geometry-aware learning for generalized few-shot object de- tection

    Jiawei Ma, Yulei Niu, Jincheng Xu, Shiyuan Huang, Guangx- ing Han, and Shih-Fu Chang. Digeo: Discriminative geometry-aware learning for generalized few-shot object de- tection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3208–3218,

  29. [38]

    Simple open-vocabulary object detection

    Matthias Minderer, Alexey Gritsenko, Austin Stone, Maxim Neumann, Dirk Weissenborn, Alexey Dosovitskiy, Aravindh Mahendran, Anurag Arnab, Mostafa Dehghani, Zhuoran Shen, et al. Simple open-vocabulary object detection. In European conference on computer vision , pages 728–755. ...

    2022

  30. [39]

    Scal- ing open-vocabulary object detection

    Matthias Minderer, Alexey Gritsenko, and Neil Houlsby. Scal- ing open-vocabulary object detection. Advances in Neural Information Processing Systems, 36:72983–73007, 2023. 6, 7

    2023

  31. [40]

    Defrcn: Decoupled faster r-cnn for few-shot object detection

    Limeng Qiao, Yuxuan Zhao, Zhiyuan Li, Xi Qiu, Jianan Wu, and Chi Zhang. Defrcn: Decoupled faster r-cnn for few-shot object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 8681–8690, 2021. 1, 2, 6, 7

    2021

  32. [41]

    Language models are unsuper- vised multitask learners

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsuper- vised multitask learners. OpenAI blog, 1(8):9, 2019. 3

    2019

  33. [42]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  34. [43]

    Adaptive multi-task learning for few-shot object detection

    Yan Ren, Yanling Li, and Adams Wai-Kin Kong. Adaptive multi-task learning for few-shot object detection. InEuropean Conference on Computer Vision, pages 297–314. Springer,

  35. [44]

    Objects365: A large-scale, high-quality dataset for object detection

    Shuai Shao, Zeming Li, Tianyuan Zhang, Chao Peng, Gang Yu, Xiangyu Zhang, Jing Li, and Jian Sun. Objects365: A large-scale, high-quality dataset for object detection. In Pro- ceedings of the IEEE/CVF international conference on com- puter vision, pages 8430–8439, 2019. 1

    2019

  36. [45]

    Few- shot adaptive faster r-cnn

    Tao Wang, Xiaopeng Zhang, Li Yuan, and Jiashi Feng. Few- shot adaptive faster r-cnn. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 7173–7182, 2019. 1, 6

    2019

  37. [47]

    Frustratingly simple few-shot object detection

    Xin Wang, Thomas E Huang, Trevor Darrell, Joseph E Gon- zalez, and Fisher Yu. Frustratingly simple few-shot object detection. arXiv preprint arXiv:2003.06957, 2020. 2

    2003 arXiv

  38. [48]

    Snida: Unlocking few-shot object detection with non- linear semantic decoupling augmentation

    Yanjie Wang, Xu Zou, Luxin Yan, Sheng Zhong, and Jiahuan Zhou. Snida: Unlocking few-shot object detection with non- linear semantic decoupling augmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12544–12553, 2024. 2, 6, 7

    2024

  39. [49]

    Multi- scale positive sample refinement for few-shot object detection

    Jiaxi Wu, Songtao Liu, Di Huang, and Yunhong Wang. Multi- scale positive sample refinement for few-shot object detection. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XVI 16, pages 456–472. Springer, 2020. 6, 7

    2020

  40. [50]

    Multi-faceted distillation of base-novel commonality for few-shot object detection

    Shuang Wu, Wenjie Pei, Dianwen Mei, Fanglin Chen, Jian- dong Tian, and Guangming Lu. Multi-faceted distillation of base-novel commonality for few-shot object detection. In European Conference on Computer Vision, pages 578–594. Springer, 2022. 2, 6, 7

    2022

  41. [51]

    Cora: Adapting clip for open-vocabulary detection with region prompting and anchor pre-matching

    Xiaoshi Wu, Feng Zhu, Rui Zhao, and Hongsheng Li. Cora: Adapting clip for open-vocabulary detection with region prompting and anchor pre-matching. In Proceedings of the IEEE/CVF conference on computer vision and pattern recog- nition, pages 7031–7040, 2023. 3

    2023

  42. [52]

    Generating fea- tures with increased crop-related diversity for few-shot ob- ject detection

    Jingyi Xu, Hieu Le, and Dimitris Samaras. Generating fea- tures with increased crop-related diversity for few-shot ob- ject detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19713– 19722, 2023. 1, 3, 6, 7

    2023

  43. [53]

    Multi-modal queried object detection in the wild

    Yifan Xu, Mengdan Zhang, Chaoyou Fu, Peixian Chen, Xi- aoshan Yang, Ke Li, and Changsheng Xu. Multi-modal queried object detection in the wild. In Proceedings of the 37th International Conference on Neural Information Pro- cessing Systems, pages 4452–4469, 2023. 1, 3, 5, 6, 7

    2023

  44. [54]

    Deeplesion: Automated deep mining, categorization and detection of significant radiology image findings us- ing large-scale clinical lesion annotations

    Ke Yan, Xiaosong Wang, Le Lu, and Ronald M Sum- mers. Deeplesion: Automated deep mining, categorization and detection of significant radiology image findings us- ing large-scale clinical lesion annotations. arXiv preprint arXiv:1710.01766, 2017. 5

    arXiv

  45. [55]

    Meta r-cnn: Towards general solver for instance-level low-shot learning

    Xiaopeng Yan, Ziliang Chen, Anni Xu, Xiaoxi Wang, Xi- aodan Liang, and Liang Lin. Meta r-cnn: Towards general solver for instance-level low-shot learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 9577–9586, 2019. 7

    2019

  46. [56]

    Meta-detr: Image-level few-shot detection with inter-class correlation exploitation

    Gongjie Zhang, Zhipeng Luo, Kaiwen Cui, Shijian Lu, and Eric P Xing. Meta-detr: Image-level few-shot detection with inter-class correlation exploitation. IEEE transactions on pat- tern analysis and machine intelligence, 45(11):12832–12843,

  47. [57]

    Detect every thing with few examples

    Xinyu Zhang, Yuting Wang, and Abdeslam Boularias. Detect every thing with few examples. arXiv preprint arXiv:2309.12969, 2023. 7

    2023 arXiv

  48. [58]

    Vlm-guided explicit-implicit complementary novel class semantic learning for few-shot object detection

    Taijin Zhao, Heqian Qiu, Yu Dai, Lanxiao Wang, Hefei Mei, Fanman Meng, Qingbo Wu, and Hongliang Li. Vlm-guided explicit-implicit complementary novel class semantic learning for few-shot object detection. Expert Systems with Applica- tions, 256:124926, 2024. 1, 3, 6, 7

    2024

  49. [59]

    Scene-adaptive and region-aware multi-modal prompt for open vocabulary object detection

    Xiaowei Zhao, Xianglong Liu, Duorui Wang, Yajun Gao, and Zhide Liu. Scene-adaptive and region-aware multi-modal prompt for open vocabulary object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16741–16750, 2024. 3

    2024

  50. [60]

    Regionclip: Region-based language-image pretraining

    Yiwu Zhong et al. Regionclip: Region-based language-image pretraining. In CVPR, pages 16793–16803, 2022. 3, 1

    2022

  51. [61]

    Conditional prompt learning for vision-language models

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Conditional prompt learning for vision-language models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16816–16825, 2022. 3

    2022

  52. [62]

    Learning to prompt for vision-language models

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to prompt for vision-language models. Interna- tional Journal of Computer Vision, 130(9):2337–2348, 2022. 3 Visual Textualization for Image Prompted Object Detection Supplementary Material

    2022

  53. [63]

    fully connected (fc) + ReLU

    Implementation details of VisTex-DINO and other comparison methods We also implemented VisTex-OVLM on GroundingDINO-T [33], denoted as VisTex-DINO. The MSTB design mirrors that of VisTex-GLIP, employing two "fully connected (fc) + ReLU" layers. Based on the feature size and sc...

  54. [64]

    4.3 of the main text, we provide detailed transfer results on the ODinW13 subsets [ 31] in Tab

    Performance on ODinW13 subsets Following Sec. 4.3 of the main text, we provide detailed transfer results on the ODinW13 subsets [ 31] in Tab. 10. ODinW13 [29] is composed of 13 subsets from ODinW35, spanning specialized natural domains such as aquarium species, surgical instru...

  55. [65]

    Compatibility experiments on RegionCLIP and FIBER We evaluated VisTex on RegionCLIP [60] under a one-shot setting using the Open-V ocabulary COCO and LVIS bench- mark, where base and novel categories are disjoint (Tab. 6). The zero-shot (ZS) results were adopted from the origi...

  56. [66]

    8, we report the computational overhead for process- ing one image using GLIP-L on RTX3090 with one support image, comparing it to MQ-Det and GLIP-FF

    Computational Overhead and Preprocess Time In Tab. 8, we report the computational overhead for process- ing one image using GLIP-L on RTX3090 with one support image, comparing it to MQ-Det and GLIP-FF. After an ini- tial preprocessing step on the support image, textualized vis...

  57. [67]

    BG blur" technique performs best. It high- lights the target object while preserving some background, unlike

    More ablation and visualization results 10.1. Multi-scale textualizing block We assessed the impact of multi-scale textualization and the parameter-sharing strategy (MSTB sharing), as shown in Tab. 11. With scales indexed by j (e.g., "0" represents the H 20 · W 20 scale), resu...

  58. [68]

    Base- line

    Output visualization for real-world down- stream tasks Fig. 8 and Fig. 6 present the output visualizations for real- world downstream tasks, including ODinW13 subsets and medical datasets (MoNu, LIDC, and Deeplesion). The visualizations in Fig. 8 adhere to the settings de- scr...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.