Pith. sign in

REVIEW 4 major objections 4 minor 1 cited by

Visual Modality Prompt for Adapting Vision-Language Object Detectors

T0 review · 4 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read ModPrompt claims that a small image-conditioned encoder-decoder, added to the input of a frozen vision-language detector, can translate infrared and depth images into a representation the detector handles well, reaching near…

desk verdict A plausible parameter-efficient modality-adaptation method for frozen vision-language detectors, but the reported numbers are inconsistent and the attribution of the gains is not yet established. read the letter →

arxiv 2412.00622 v2 pith:INYOHZHE submitted 2024-12-01 cs.CV cs.AIcs.LG

Heitor R. Medeiros , Atif Belal , Srikanth Muralidharan , Eric Granger , Marco Pedersoli This is my paper · ORCID
classification cs.CVcs.AIcs.LG
keywords visualpromptmodalityadaptationopen-vocabularyobjectdetectioninfrareddepthzero-shotknowledgepreservationencoder-decodertextembeddingresidual
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Open-vocabulary object detectors trained on ordinary RGB images lose accuracy when tested on infrared or depth images, and fine-tuning them on the new modality erases their ability to detect the original categories. This paper proposes ModPrompt, a way to adapt such detectors by learning a small image-to-image translator that is added to each input image while every weight of the detector stays frozen. The translator is trained only by the detector's own detection loss, so no reconstruction or alignment between modalities is needed. On infrared benchmarks the results approach full fine-tuning, and on depth data the gains are large, while the detector's zero-shot performance on RGB is preserved exactly. A companion text-side residual, called MPDR, tunes the class embeddings without overwriting the originals, which can be restored by zero-masking at test time.

What carries the argument

The central object is the input-conditioned visual prompt $h_\vartheta(x)$: an encoder-decoder network shaped like a U-Net whose output is constrained to three channels in $[0,1]$, trained solely by the frozen detector's detection loss once it is added to the image. The key identity is the additive combination $x + h_\vartheta(x)$ used as the detector input, with no auxiliary reconstruction or alignment loss. The second piece is MPDR, a learnable residual vector added to precomputed text embeddings for the target classes; it is trained together with $h_\vartheta$ but kept separate from the frozen embeddings so the original zero-shot text knowledge survives and can be restored by zero-masking. Static prompts fail, the paper argues, because they apply the same translation to every image; conditioning the prompt on the input is what lets the translation handle large modality shifts.

What would settle it

Train ModPrompt on a target dataset with the ground-truth box labels randomly shuffled or replaced by constant boxes; if the reported gains over zero-shot persist, the improvement is not coming from detection-supervised translation, and the central mechanism fails. Alternatively, evaluate the adapted detector on the original COCO zero-shot set with MPDR zero-masked; any drop below the untrained zero-shot numbers would contradict the knowledge-preservation claim.

Watch

Extended reading notes

Core claim

The paper claims that the right place to adapt a vision-language detector to a new visual modality is the input image itself: a U-Net-style encoder-decoder $h_\vartheta$ reads the target-modality image $x$ and produces a pseudo-RGB prompt that is added to $x$, and the frozen detector $f_\theta$ is then scored against the ground-truth boxes with the ordinary detection loss $L_{\mathrm{det}}(f_\theta(x + h_\vartheta(x)), Y)$. Because the prompt is conditioned on each input image rather than being a fixed patch, it can translate infrared or depth content into a form the RGB-pre-trained detector can interpret. The paper further claims that adapting the text branch as a decoupled residual (MPDR) added to the frozen class embeddings improves the target-modality results and, unlike direct embedding tuning, does not destroy the original zero-shot knowledge, since zero-masking the residual restores the original embeddings. Empirically the paper reports that on the LLVIP infrared dataset with YOLO-World, ModPrompt reaches 92.80 AP50 against 81.00 for zero-shot and 97.43 for full fine-tuning, and on NYUv2 depth it raises AP50 from 4.80 to 37.17, with corresponding gains for Grounding DINO.

Load-bearing premise

The load-bearing premise is that the frozen detector's detection loss, computed on the summed image, sends a strong enough training signal to teach the prompt network to translate the new modality, even though no reconstruction or alignment loss is used and the summed image is not guaranteed to look like an ordinary RGB image.

Editorial extensions

If this is right

  • With only about 3.08 million trainable parameters, ModPrompt on YOLO-World reaches LLVIP infrared AP50 of 92.80, up from 81.00 zero-shot and within about 4.6 points of full fine-tuning at 97.43, while keeping COCO zero-shot AP50 at 51.90.
  • On NYUv2 depth, ModPrompt raises YOLO-World AP50 from 4.80 to 37.17, a large improvement over zero-shot, though still below the full-fine-tuning level of 49.90.
  • MPDR adds gains for most of the tested visual prompt strategies while preserving the original knowledge, and zero-masking it at inference returns the detector to its exact zero-shot text embeddings.
  • ModPrompt is backbone-agnostic: it works on both the CNN-based YOLO-World and the transformer-based Grounding DINO with the same additive prompt recipe.
  • Static visual prompts such as fixed, random, padding, and weight-map patches can perform worse than zero-shot on large modality shifts, which supports the paper's claim that input-conditioned prompts are necessary for modality adaptation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the same detection-loss-only recipe should extend to other large modality gaps, such as radar or medical imaging, because Equation 3 does not depend on infrared or depth specifically.
  • Beyond the paper: because the prompt is additive and the detector stays frozen, ModPrompt could be combined with test-time adaptation or unsupervised losses to reduce the need for target labels, an extension the authors list as future work.
  • Beyond the paper: the near-full-fine-tuning result on LLVIP suggests that RGB pre-training already encodes enough structure for infrared pedestrian detection, so even lighter prompt backbones may suffice, consistent with the MobileNet-ResNet ablation in the paper.
  • Beyond the paper: MPDR-style decoupled residual tuning could be ported to other vision-language tasks, such as classification or retrieval, whenever per-domain adaptation must not overwrite the original embeddings.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes ModPrompt, an encoder-decoder visual prompt that transforms input images from a target modality (infrared or depth) into a pseudo-RGB representation fed to a frozen vision-language object detector (YOLO-World or Grounding DINO), trained end-to-end with the detection loss. It additionally introduces MPDR, a learnable residual added to precomputed text embeddings, intended to preserve the detector's zero-shot text knowledge while adapting to the new modality. Experiments on LLVIP, FLIR, and NYUv2 compare ModPrompt against zero-shot, head/full fine-tuning, and several pixel-level visual prompt baselines, and report that ModPrompt achieves performance "comparable to full fine-tuning" while preserving zero-shot capability. The paper also compares against prior modality-translation methods HalluciDet and ModTr.

Significance. If the reported results hold, the paper addresses a meaningful gap: adapting modern vision-language object detectors to new visual modalities without losing their open-vocabulary zero-shot abilities. The approach is detector-agnostic, is evaluated on two detector families and three datasets, and the authors release code, which are concrete strengths. The notion of an input-dependent, detection-loss-driven visual prompt for VLMs is a reasonable and potentially useful contribution. However, the current manuscript contains numerical inconsistencies across tables, an omitted control that prevents attributing gains to the visual prompt, and an under-specified text-residual component; these issues currently block a reliable assessment of the central claims.

major comments (4)
  1. [§4.2, Tables 1, 2, and 4] The headline result for YOLO-World on LLVIP is not consistently reported: Table 1 gives ModPrompt AP50 92.80, Table 2 gives 96.60, and Table 4 gives 95.63. Table 4's average 73.77 and the +7.32 gain over zero-shot are computed from 95.63, a value that appears in no other table. §4.2 repeats 92.80. This inconsistency directly affects the abstract's claim of "performance comparable to full fine-tuning," and the NYUv2 results in Table 1 (AP 24.93 vs FT 33.57) show a gap of 8.64 AP, which is not "comparable" in the usual sense. The authors must reconcile these numbers and qualify the claim accordingly.
  2. [§4.2, Table 2] The contribution of the learned encoder-decoder prompt hϑ from Eq. (3) is not isolated. Table 2 shows that adding MPDR to weak visual-prompt baselines produces large gains (e.g., YOLO-World LLVIP AP50: Fixed 70.30→86.60, Random 60.13→86.43, Padding 79.87→83.57). Since MPDR is trained jointly with ModPrompt in all reported configurations, the observed gains may be largely due to the text residual rather than the visual translation that the paper's central claim emphasizes. A control experiment with MPDR alone applied to the original zero-shot image (i.e., hϑ(x)=0) is necessary to attribute the improvement to the visual prompt component.
  3. [§3.4, Eq. (4)] The MPDR objective is under-specified. Eq. (4) defines the final cost as Cmp(ϑ) + Ctp(ϕ) with Ctp defined in the supplementary material on text inputs, but §3.4 states that instead of the online text cost, the method works on the text-embedding space. The actual training objective for the residual added to precomputed embeddings is never written down, nor is the zero-masking deactivation operation defined formally. Without this, the reader cannot determine what exactly is optimized, how the residual is scaled, or why zero-masking recovers the original embedding. A precise formulation is needed for reproducibility.
  4. [§3.2, Eq. (3)] The input-domain constraint is unclear. The text says the decoder's last layer is constrained to produce values in [0,1], but the prompted input is x + hϑ(x). If x is an image with pixel values in [0,1], the sum can exceed 1; if x is in [0,255], the sum is even further from the expected input range. No normalization or clipping is specified. Since the frozen detector was trained on RGB images, the distribution of x + hϑ(x) is load-bearing for the method's validity. Please specify how the summed input is normalized or why the out-of-range values are unproblematic.
minor comments (4)
  1. [§4.3] The abbreviation is inconsistent: the paragraph "MPDR Knowledge Preservation mechanism" twice writes "MDPR" instead of "MPDR."
  2. [§4.4] The text says "Figure 3 shows the visualization of ModPrompt on YOLO-World," but Figure 3 in the main text is the text-prompt tuning schematic; the qualitative detection examples appear to be Figure 5. The reference should be corrected.
  3. [§4.3] The ablation section says "we tested various patch sizes for each of the visual prompt methods and reported the performance in Table 8," but the main-text table with these variations is Table 3; Table 8 is in the supplementary material. This cross-reference is confusing.
  4. [§4.3] The sentence reporting the direct-embedding adaptation without MPDR states that performance was "close to MDPR" and zero-shot knowledge was lost, but gives no numbers. Since this directly motivates the MPDR design, a quantitative comparison belongs in the main text or a clearly referenced supplementary table.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: ModPrompt is an empirical supervised adaptation pipeline, and the paper's self-citations are non-load-bearing comparisons.

full rationale

The paper's derivation chain is an empirical supervised learning pipeline rather than a formal derivation that reduces to its inputs. Equation (3) trains the encoder-decoder prompt hϑ by minimizing the frozen detector's detection loss Ldet(fθ(x + hϑ(x)), Y) on labeled target-modality data, and Equation (4) jointly trains the MPDR text-embedding residual with the same detection loss. Neither equation defines the target result in terms of itself: the reported AP gains on LLVIP, FLIR, and NYUv2 are measured on held-out test images after training, so they are not forced by construction. The zero-shot preservation claim is supported by the frozen detector weights and the residual form of MPDR, but the COCO column in Table 4 is still an empirical check rather than a tautology, because applying a learned input transform could in principle change COCO performance; the paper reports that it does not. Self-citations, including ModTr [31] and HalluciDet [33], are used only as comparison baselines or contextual background, not as evidence for the central claim, so they do not constitute load-bearing circularity. No uniqueness theorem is imported from the authors' prior work, no ansatz is smuggled in through a self-citation, and no known result is merely renamed. The inconsistencies in the reported ModPrompt numbers (for example, YOLO-World LLVIP AP50 appears as 92.80 in Table 1, 96.60 in Table 2, and 95.63 in Table 4) are reproducibility and attribution concerns rather than circularity, and the missing MPDR-only control affects attribution of the visual-prompt contribution without making the derivation circular.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The method is an empirical supervised adaptation procedure. No new entities are introduced. The central claim rests on the standard assumption that detection loss gradients through a frozen detector can train an input-level prompt, on the unstated input-normalization choice, and on the assumption that additive text-embedding residuals can be zero-masked to restore the original zero-shot behavior. Hand-chosen hyperparameters and the choice of prompt backbone are free parameters that affect the reported AP.

free parameters (4)
  • Prompt encoder-decoder backbone choice = MobileNet or ResNet (U-Net style)
    The choice of pretrained encoder for the translation network is a design decision that affects the target-modality AP (Tables 3 and 8); no selection rule is given.
  • Learning rate and optimizer = AdamW, lr = 2e-4 (YOLO-World); AdamW, lr = 1e-4 (Grounding DINO)
    Hand-chosen hyperparameters; the paper gives no sensitivity analysis.
  • Training epochs and batch size = 80 epochs, batch 8 (YOLO-World); 60 epochs, batch 16 (Grounding DINO)
    Hand-chosen values used across all reported comparisons; no ablation is shown.
  • MPDR residual size or scaling = Not specified
    The magnitude and initialization of the learned text-embedding residual are not reported, yet they affect the MPDR gains in Table 2.
assumptions (3)
  • domain assumption The frozen detector's loss Ldet provides a sufficiently informative gradient to train the prompt hϑ for all target images.
    Invoked in Eq. 3 when optimizing Cmp(ϑ) without any auxiliary alignment or reconstruction loss.
  • domain assumption Adding the prompt output hϑ(x) to the input x (sum in Eq. 3) keeps the image in a distribution the frozen RGB detector can interpret, after unspecified clipping or normalization.
    The sum x + hϑ(x) with hϑ in [0,1] can exceed the standard [0,1] input range; the paper does not state the preprocessing.
  • ad hoc to paper Text embeddings for target classes can be precomputed offline from the frozen text encoder, and adding a learnable residual (MPDR) preserves the detector's matching behavior when the residual is zero-masked.
    Described in Sec. 3.3; no formal statement or proof is given that zero-masking recovers the original embedding exactly.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Visual Modality Prompt for Adapting Vision-Language Object Detectors." pith.science (2026). https://pith.science/paper/INYOHZHE

@misc{pith2026241200622,
  author       = {Pith},
  title        = {Pith review of: Visual Modality Prompt for Adapting Vision-Language Object Detectors},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/INYOHZHE}},
  note         = {Machine review of arXiv:2412.00622}
}
read the original abstract

The zero-shot performance of object detectors degrades when tested on different modalities, such as infrared and depth. While recent work has explored image translation techniques to adapt detectors to new modalities, these methods are limited to a single modality and apply only to traditional detectors. Recently, vision-language detectors, such as YOLO-World and Grounding DINO, have shown promising zero-shot capabilities, however, they have not yet been adapted for other visual modalities. Traditional fine-tuning approaches compromise the zero-shot capabilities of the detectors. The visual prompt strategies commonly used for classification with vision-language models apply the same linear prompt translation to each image, making them less effective. To address these limitations, we propose ModPrompt, a visual prompt strategy to adapt vision-language detectors to new modalities without degrading zero-shot performance. In particular, an encoder-decoder visual prompt strategy is proposed, further enhanced by the integration of inference-friendly modality prompt decoupled residual, facilitating a more robust adaptation. Empirical benchmarking results show our method for modality adaptation on two vision-language detectors, YOLO-World and Grounding DINO, and on challenging infrared (LLVIP, FLIR) and depth (NYUv2) datasets, achieving performance comparable to full fine-tuning while preserving the model's zero-shot capability. Code available at: https://github.com/heitorrapela/ModPrompt.

Figures

Figures reproduced from arXiv: 2412.00622 by the authors.

Figure 1
Figure 1. Detections of different approaches across modalities: LLVIP and FLIR datasets (infrared) and NYUv2 (depth). Each column corresponds to a different approach: (a) GT (Ground Truth): Shows in yellow the ground-truth bounding boxes for objects. (b) Zero￾Shot: Displays detections (in red) from a zero-shot model. This model has missed several detections and some inaccurate boxes without specific tuning. (c) Visual Prompt:… view at source ↗
Figure 2
Figure 2. Strategies to adapt object detectors to new modalities: [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Our proposed strategy for text-prompt tuning: [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Detection performance on LLVIP dataset of different [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Detections for YOLO-World for the different approaches: First two rows for LLVIP (infrared), and last two rows for NYUv2 (depth). Each column corresponds to a different approach: (a) GT (Ground Truth): Shows in yellow the ground-truth bounding boxes for objects. (b) Ze…
Figure 1
Figure 1. Figure 1: Detections of different approaches across modalities: LLVIP and FLIR datasets (infrared) and NYUv2 (depth). Each column corresponds to a different approach: (a) GT (Ground Truth): Shows in yellow the ground-truth bounding boxes for objects. (b) Zero￾Shot: Displays dete…
Figure 2
Figure 2. Figure 2: Detection performance on FLIR-IR dataset of different [PITH_FULL_IMAGE:figures/full_fig_p012_2.png]
Figure 3
Figure 3. Figure 3: Detections of different approaches across modalities for YOLO-World: NYUv2 (depth) and FLIR (infrared). Each row corresponds to a different approach: GT (Ground Truth): Shows in yellow the ground-truth bounding boxes for objects. ZS (Zero-Shot): Displays detections (in…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. VLOD-TTA: Test-Time Adaptation of Vision-Language Object Detectors

    cs.CV 2025-10 conditional novelty 6.0 of 10

    An IoU-weighted entropy objective and image-conditioned prompt selection adapt YOLO-World and Grounding DINO at test time, improving robustness on style, weather, low-light, and corruption shifts without labels.

Reference graph

Works this paper leans on

56 extracted references · 32 canonical work pages · cited by 1 Pith paper

  1. [1]

    Exploring visual prompts for adapting large- scale models

    Hyojin Bahng, Ali Jahanian, Swami Sankaranarayanan, and Phillip Isola. Exploring visual prompts for adapting large- scale models. arXiv preprint arXiv:2203.17274 , 2022. 2, 3

    arXiv 2022

  2. [2]

    Zero-shot object detection

    Ankan Bansal, Karan Sikka, Gaurav Sharma, Rama Chel- lappa, and Ajay Divakaran. Zero-shot object detection. In Proceedings of the European conference on computer vision (ECCV), pages 384–400, 2018. 2

    work page 2018

  3. [3]

    A systematic literature review on object detection using near infrared and thermal images

    Nicolas Bustos, Mehrsa Mashhadi, Susana K Lai-Yuen, Sudeep Sarkar, and Tapas K Das. A systematic literature review on object detection using near infrared and thermal images. Neurocomputing, page 126804, 2023. 2

    work page 2023

  4. [4]

    Multimodal object detection by channel switching and spatial attention

    Yue Cao, Junchi Bin, Jozsef Hamari, Erik Blasch, and Zheng Liu. Multimodal object detection by channel switching and spatial attention. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition , pages 403–411, 2023. 6

    work page 2023

  5. [5]

    MMDetection: Open mmlab detection toolbox and benchmark

    Kai Chen, Jiaqi Wang, Jiangmiao Pang, Yuhang Cao, Yu Xiong, Xiaoxiao Li, Shuyang Sun, Wansen Feng, Ziwei Liu, Jiarui Xu, Zheng Zhang, Dazhi Cheng, Chenchen Zhu, Tian- heng Cheng, Qijie Zhao, Buyu Li, Xin Lu, Rui Zhu, Yue Wu, Jifeng Dai, Jingdong Wang, Jianping Shi, Wanli Ouyang, Chen Change Loy, and Dahua Lin. MMDetection: Open mmlab detection toolbox and...

    arXiv 1906

  6. [6]

    Yolo-world: Real-time open-vocabulary object detection

    Tianheng Cheng, Lin Song, Yixiao Ge, Wenyu Liu, Xing- gang Wang, and Ying Shan. Yolo-world: Real-time open-vocabulary object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16901–16911, 2024. 2, 3, 6, 11

    work page 2024

  7. [7]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018. 6

    arXiv 2018

  8. [8]

    Privacy-preserving person detection using low-resolution infrared cameras

    Thomas Dubail, Fidel Alejandro Guerrero Pe ˜na, Heitor Rapela Medeiros, Masih Aminbeidokhti, Eric Granger, and Marco Pedersoli. Privacy-preserving person detection using low-resolution infrared cameras. In Eu- ropean Conference on Computer Vision , pages 689–702. Springer, 2022. 2

    work page 2022

Show all 56 references
  1. [9]

    Multimodal deep learning for robust rgb-d object recognition

    Andreas Eitel, Jost Tobias Springenberg, Luciano Spinello, Martin Riedmiller, and Wolfram Burgard. Multimodal deep learning for robust rgb-d object recognition. In 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 681–687. IEEE, 2015. 2

    2015

  2. [10]

    Tood: Task-aligned one-stage object detec- tion

    Chengjian Feng, Yujie Zhong, Yu Gao, Matthew R Scott, and Weilin Huang. Tood: Task-aligned one-stage object detec- tion. In 2021 IEEE/CVF International Conference on Com- puter Vision (ICCV), pages 3490–3499. IEEE Computer So- ciety, 2021. 11

    2021

  3. [11]

    Clip-adapter: Better vision-language models with feature adapters

    Peng Gao, Shijie Geng, Renrui Zhang, Teli Ma, Rongyao Fang, Yongfeng Zhang, Hongsheng Li, and Yu Qiao. Clip-adapter: Better vision-language models with feature adapters. International Journal of Computer Vision, 132(2): 581–595, 2024. 3

    2024

  4. [12]

    Generative adversarial nets

    Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in neural information processing systems, 27, 2014. 3

    2014

  5. [13]

    Open-vocabulary object detection via vision and language knowledge distillation

    Xiuye Gu, Tsung-Yi Lin, Weicheng Kuo, and Yin Cui. Open-vocabulary object detection via vision and language knowledge distillation. arXiv preprint arXiv:2104.13921 ,

    arXiv

  6. [14]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 7, 12

    2016

  7. [15]

    Cnn- based thermal infrared person detection by domain adapta- tion

    Christian Herrmann, Miriam Ruf, and J ¨urgen Beyerer. Cnn- based thermal infrared person detection by domain adapta- tion. In Autonomous Systems: Sensors, Vehicles, Security, and the Internet of Everything , page 1064308. International Society for Optics and Photonics, 2018. 2

    2018

  8. [16]

    Mobilenets: Efficient convolu- tional neural networks for mobile vision applications

    Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco An- dreetto, and Hartwig Adam. Mobilenets: Efficient convolu- tional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861, 2017. 7, 12

    2017 arXiv

  9. [17]

    Image-to-image translation with conditional adver- sarial networks

    Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with conditional adver- sarial networks. In Computer Vision and Pattern Recognition (CVPR), 2017 IEEE Conference on, 2017. 4

    2017

  10. [18]

    Multimodal computer vision framework for human assistive robotics

    Eugenio Ivorra, Mario Ortega, Mariano Alca ˜niz, and Nicol´as Garcia-Aracil. Multimodal computer vision framework for human assistive robotics. In 2018 Workshop on Metrology for Industry 4.0 and IoT, pages 1–5. IEEE, 2018. 2

    2018

  11. [19]

    Vi- sual prompt tuning

    Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, and Ser-Nam Lim. Vi- sual prompt tuning. In European Conference on Computer Vision, pages 709–727. Springer, 2022. 2, 3

    2022

  12. [20]

    Ultralyt- ics yolov8

    Glenn Jocher, Ayush Chaurasia, and Jing Qiu. Ultralyt- ics yolov8. https://github.com/ultralytics/ ultralytics, 2023. 2, 11

    2023

  13. [21]

    Mdetr – mod- ulated detection for end-to-end multi-modal understanding,

    Aishwarya Kamath, Mannat Singh, Yann LeCun, Gabriel Synnaeve, Ishan Misra, and Nicolas Carion. Mdetr – mod- ulated detection for end-to-end multi-modal understanding,

  14. [22]

    Auto-encoding varia- tional bayes, 2022

    Diederik P Kingma and Max Welling. Auto-encoding varia- tional bayes, 2022. 3

    2022

  15. [23]

    The power of scale for parameter-efficient prompt tuning, 2021

    Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter-efficient prompt tuning, 2021. 2

    2021

  16. [24]

    Grounded language-image pre-training

    Liunian Harold Li, Pengchuan Zhang, Haotian Zhang, Jian- wei Yang, Chunyuan Li, Yiwu Zhong, Lijuan Wang, Lu Yuan, Lei Zhang, Jenq-Neng Hwang, et al. Grounded language-image pre-training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, page...

    2022

  17. [25]

    Yolo-firi: Improved yolov5 for infrared image object detection

    Shasha Li, Yongjun Li, Yao Li, Mengjun Li, and Xiaorong Xu. Yolo-firi: Improved yolov5 for infrared image object detection. IEEE access, 9:141861–141875, 2021. 2

    2021

  18. [26]

    Bevdepth: Acquisition of reliable depth for multi-view 3d object detec- tion

    Yinhao Li, Zheng Ge, Guanyi Yu, Jinrong Yang, Zengran Wang, Yukang Shi, Jianjian Sun, and Zeming Li. Bevdepth: Acquisition of reliable depth for multi-view 3d object detec- tion. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 1477–1485, 2023. 2 15

    2023

  19. [27]

    Learning object-language alignments for open-vocabulary object de- tection

    Chuang Lin, Peize Sun, Yi Jiang, Ping Luo, Lizhen Qu, Gho- lamreza Haffari, Zehuan Yuan, and Jianfei Cai. Learning object-language alignments for open-vocabulary object de- tection. arXiv preprint arXiv:2211.14843, 2022. 3

    2022 arXiv

  20. [28]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision , pages 740–755. Springer, 2014. 2, 6

    2014

  21. [29]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection

    Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. arXiv preprint arXiv:2303.05499, 2023. 2, 3

    2023 arXiv

  22. [30]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pages 10012–10022, 2021. 6

    2021

  23. [31]

    Modality translation for object detection adapta- tion without forgetting prior knowledge

    Heitor Rapela Medeiros, Masih Aminbeidokhti, Fidel Guer- rero Pena, David Latortue, Eric Granger, and Marco Ped- ersoli. Modality translation for object detection adapta- tion without forgetting prior knowledge. arXiv preprint arXiv:2404.01492, 2024. 3, 4, 7, 12

    2024 arXiv

  24. [32]

    Mipa: Mixed patch infrared-visible modality ag- nostic object detection

    Heitor R Medeiros, David Latortue, Eric Granger, and Marco Pedersoli. Mipa: Mixed patch infrared-visible modality ag- nostic object detection. arXiv preprint arXiv:2404.18849 ,

    arXiv

  25. [33]

    Hallucidet: Hallucinating rgb modality for person de- tection through privileged information

    Heitor Rapela Medeiros, Fidel A Guerrero Pena, Masih Am- inbeidokhti, Thomas Dubail, Eric Granger, and Marco Ped- ersoli. Hallucidet: Hallucinating rgb modality for person de- tection through privileged information. InProceedings of the IEEE/CVF Winter Conference on Applicatio...

    2024

  26. [34]

    Benchmarking ro- bustness in object detection: Autonomous driving when win- ter is coming

    Claudio Michaelis, Benjamin Mitzkus, Robert Geirhos, Evgenia Rusak, Oliver Bringmann, Alexander S Ecker, Matthias Bethge, and Wieland Brendel. Benchmarking ro- bustness in object detection: Autonomous driving when win- ter is coming. arXiv preprint arXiv:1907.07484, 2019. 2

    1907 arXiv

  27. [35]

    Infragan: A gan ar- chitecture to transfer visible images to infrared domain

    Mehmet Akif ¨Ozkano˘glu and Sedat Ozer. Infragan: A gan ar- chitecture to transfer visible images to infrared domain. Pat- tern Recognition Letters, 155:69–76, 2022. 2

    2022

  28. [36]

    Image-to-image translation: Methods and applications,

    Yingxue Pang, Jianxin Lin, Tao Qin, and Zhibo Chen. Image-to-image translation: Methods and applications,

  29. [37]

    Deep learning in robotics: a review of recent research

    Harry A Pierson and Michael S Gashler. Deep learning in robotics: a review of recent research. Advanced Robotics, 31 (16):821–835, 2017. 2

    2017

  30. [38]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  31. [39]

    A review on object detection in unmanned aerial vehicle surveillance

    Anitha Ramachandran and Arun Kumar Sangaiah. A review on object detection in unmanned aerial vehicle surveillance. International Journal of Cognitive Computing in Engineer- ing, 2:215–228, 2021. 2

    2021

  32. [40]

    U- net: Convolutional networks for biomedical image segmen- tation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U- net: Convolutional networks for biomedical image segmen- tation. In International Conference on Medical image com- puting and computer-assisted intervention , pages 234–241. Springer, 2015. 5

    2015

  33. [41]

    Objects365: A large-scale, high-quality dataset for object detection

    Shuai Shao, Zeming Li, Tianyuan Zhang, Chao Peng, Gang Yu, Xiangyu Zhang, Jing Li, and Jian Sun. Objects365: A large-scale, high-quality dataset for object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2019. 6

    2019

  34. [42]

    Indoor segmentation and support inference from rgbd images

    Nathan Silberman, Derek Hoiem, Pushmeet Kohli, and Rob Fergus. Indoor segmentation and support inference from rgbd images. In Computer Vision–ECCV 2012: 12th Eu- ropean Conference on Computer Vision, Florence, Italy, Oc- tober 7-13, 2012, Proceedings, Part V 12 , pages 746–760...

    2012

  35. [43]

    Machine learning, social learning and the gov- ernance of self-driving cars

    Jack Stilgoe. Machine learning, social learning and the gov- ernance of self-driving cars. Social studies of science, 48(1): 25–56, 2018. 2

    2018

  36. [44]

    Improving rgb-infrared object detec- tion by reducing cross-modality redundancy

    Qingwang Wang, Yongke Chi, Tao Shen, Jian Song, Zifeng Zhang, and Yan Zhu. Improving rgb-infrared object detec- tion by reducing cross-modality redundancy. Remote Sens- ing, 14(9):2020, 2022. 2

    2020

  37. [45]

    Multi-modal deep feature learning for rgb-d object detection

    Xiangyang Xu, Yuncheng Li, Gangshan Wu, and Jiebo Luo. Multi-modal deep feature learning for rgb-d object detection. Pattern Recognition, 72:300–313, 2017. 2

    2017

  38. [46]

    Deepinteraction: 3d object detection via modality interaction

    Zeyu Yang, Jiaqi Chen, Zhenwei Miao, Wei Li, Xiatian Zhu, and Li Zhang. Deepinteraction: 3d object detection via modality interaction. Advances in Neural Information Pro- cessing Systems, 35:1992–2005, 2022. 2

    1992

  39. [47]

    Task residual for tuning vision-language models

    Tao Yu, Zhihe Lu, Xin Jin, Zhibo Chen, and Xinchao Wang. Task residual for tuning vision-language models. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10899–10909, 2023. 3, 5

    2023

  40. [48]

    Open-vocabulary detr with conditional matching

    Yuhang Zang, Wei Li, Kaiyang Zhou, Chen Huang, and Chen Change Loy. Open-vocabulary detr with conditional matching. In European Conference on Computer Vision , pages 106–122. Springer, 2022. 2

    2022

  41. [49]

    Multispectral fusion for object detection with cyclic fuse-and-refine blocks

    Heng Zhang, Elisa Fromont, S ´ebastien Lef `evre, and Bruno Avignon. Multispectral fusion for object detection with cyclic fuse-and-refine blocks. In 2020 IEEE International Conference on Image Processing (ICIP) , pages 276–280. IEEE, 2020. 6

    2020

  42. [50]

    Dino: Detr with improved denoising anchor boxes for end-to-end object detection

    Hao Zhang, Feng Li, Shilong Liu, Lei Zhang, Hang Su, Jun Zhu, Lionel M Ni, and Heung-Yeung Shum. Dino: Detr with improved denoising anchor boxes for end-to-end object detection. arXiv preprint arXiv:2203.03605, 2022. 2

    2022 arXiv

  43. [51]

    Glipv2: Unifying localiza- tion and vision-language understanding

    Haotian Zhang, Pengchuan Zhang, Xiaowei Hu, Yen-Chun Chen, Liunian Li, Xiyang Dai, Lijuan Wang, Lu Yuan, Jenq- Neng Hwang, and Jianfeng Gao. Glipv2: Unifying localiza- tion and vision-language understanding. Advances in Neural Information Processing Systems, 35:36067–36080, 2022. 2, 3

    2022

  44. [52]

    Regionclip: Region- based language-image pretraining

    Yiwu Zhong, Jianwei Yang, Pengchuan Zhang, Chun- yuan Li, Noel Codella, Liunian Harold Li, Luowei Zhou, Xiyang Dai, Lu Yuan, Yin Li, et al. Regionclip: Region- based language-image pretraining. In Proceedings of the 16 IEEE/CVF conference on computer vision and pattern recogni...

    2022

  45. [53]

    Conditional prompt learning for vision-language mod- els

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Conditional prompt learning for vision-language mod- els. In Proceedings of the IEEE/CVF conference on com- puter vision and pattern recognition , pages 16816–16825,

  46. [54]

    Learning to prompt for vision-language models

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to prompt for vision-language models. In- ternational Journal of Computer Vision, 130(9):2337–2348,

  47. [55]

    Unpaired image-to-image translation using cycle- consistent adversarial networks

    Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-image translation using cycle- consistent adversarial networks. In Computer Vision (ICCV), 2017 IEEE International Conference on, 2017. 4

    2017

  48. [56]

    Object detection in 20 years: A survey.Proceed- ings of the IEEE, 111(3):257–276, 2023

    Zhengxia Zou, Keyan Chen, Zhenwei Shi, Yuhong Guo, and Jieping Ye. Object detection in 20 years: A survey.Proceed- ings of the IEEE, 111(3):257–276, 2023. 2 17

    2023

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.