Pith. sign in

REVIEW 4 major objections 5 minor 55 references

Zero-Shot Scene Understanding for Automatic Target Recognition Using Large Vision-Language Models

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read An open-world detector paired with a vision-language model labels military vehicles never seen in training.

desk verdict Solid empirical sweep of LVLMs for military vehicle ATR, but the open-set accuracy metric is fitted post-hoc to the test set, so the headline zero-shot numbers are not trustworthy. read the letter →

arxiv 2501.07396 v1 pith:MXXLFBRB submitted 2025-01-13 cs.CV

Yasiru Ranasinghe , Vibashan VS , James Uplinger , Celso De Melo , Vishal M. Patel This is my paper · ORCID
classification cs.CV
keywords automatictargetrecognitionzero-shotlearninglargevision-languagemodelsopen-worldobjectdetectionYOLO-worldchain-of-thoughtpromptingmilitaryvehiclethermalimagery
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes a two-stage pipeline for automatic target recognition (ATR) that separates localization from naming: an open-world detector crops candidate vehicles, and a large vision-language model (LVLM) assigns each crop a label without retraining. The authors argue this lets a system recognize military vehicle classes it has never seen across distance ranges, thermal imagery, and weather-degraded scenes. They report that the strongest API models reach roughly 58 to 73 percent accuracy depending on dataset and prompting protocol, well above a CLIP baseline, and that chain-of-thought prompting consistently helps. The intended payoff is an ATR system that can be pointed at novel categories and unknown environments and still produce useful labels.

What carries the argument

The load-bearing object is the cascaded binary-detection-then-reevaluation pipeline. YOLO-world, an open-vocabulary detector, is prompted with a single class keyword ('vehicle') so that it acts as a class-agnostic localizer, sidestepping its unreliable confidence on novel classes. The LVLM reevaluation is the recognition engine, with three protocols: open-set (free naming), closed-set (forced choice from a label list with a novel option), and chain-of-thought (attribute description before labeling). The binary keyword choice is what makes the system zero-shot, because the detector never commits to a closed vocabulary.

What would settle it

Fix the label-to-class mapping before inference: provide a predefined alias list for the eight classes in the long-range military dataset, run the open-set prompt, and score exact or fixed-list matches. If accuracy falls materially below the reported 67 to 73 percent at close range, the post-hoc keyword alignment is responsible for the headline numbers. A second check is to test on held-out vehicle classes that were not used when selecting the recurring-keyword mapping.

Watch

Extended reading notes

Core claim

The paper's central claim is that combining detection and recognition as separate modules overcomes each module's weakness: YOLO-world, prompted with the single binary keyword 'vehicle', produces bounding boxes regardless of object class, and an LVLM labels each crop. In open-set prompting the model is asked to name the specific vehicle in one response; in closed-set prompting it chooses from a provided label list plus a 'novel' option; in chain-of-thought prompting it first describes attributes and then labels. On the military long-range dataset, the best API models score about 67 to 73 percent at 1,000 to 2,000 meter ranges, and on the synthetic desert dataset about 64 to 73 percent, with accuracy falling sharply at 3,000 to 5,000 meters. The paper also shows the pipeline can discard detector false positives and that LVLMs surpass the detector itself on recognition, though the improvement on thermal imagery is smaller.

Load-bearing premise

The headline accuracy depends on choosing the correct answer key after seeing the model's free-text labels, so the reported scores measure recall under a lenient label-matching rule rather than true zero-shot naming.

Editorial extensions

If this is right

  • Novel military vehicle types can be labeled without retraining or new annotation, as long as the detector can crop them.
  • Adding an LVLM reevaluation stage lets an existing ATR pipeline discard detector false positives by checking each crop's label plausibility.
  • Chain-of-thought prompting improves recognition relative to direct labeling, especially for similar vehicle classes.
  • Performance degrades predictably with range and weather, and the gap between API and open-source models is large.
  • The same two-stage recipe should transfer to other novel object domains where a detector can localize but cannot name.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the label-to-class mapping for open-set scoring is chosen after seeing the model's outputs, the reported numbers are ceilings; a deployment with a fixed vocabulary or exact-match rule would likely score lower.
  • A natural extension is to hold out a disjoint set of vehicle classes and measure human agreement with the model's proposed names instead of matching to ground truth.
  • The pipeline's upper bound is set by detector recall during binary localization, so improving class-agnostic proposals should transfer directly to ATR accuracy.
  • The smaller models' failures on degraded scenes suggest using strong LVLMs to label crops from weak detectors, then training compact specialized models on those labels.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a two-stage pipeline for zero-shot automatic target recognition (ATR): YOLO-world is used for binary detection with the prompt keyword 'vehicle', and a large vision-language model (LVLM) labels each detected crop under one of three settings (open-set, closed-set, or chain-of-thought). The authors benchmark twelve LVLMs plus CLIP on four datasets (ADAS, AIS, DSIAC, and weather-degraded AIS) and report accuracy tables to support claims that the pipeline enables zero-shot recognition of novel military vehicle classes across modalities and distances.

Significance. If the quantitative results were valid, the paper would offer a useful, systematic comparison of LVLMs for military-vehicle recognition under domain shift, and the proposed division of labor (an open-vocabulary detector for localization, an LVLM for identification) is a reasonable design. The breadth of models and conditions, the qualitative demonstration of false-positive removal via LVLMs, and the chain-of-thought analysis are potentially valuable contributions. However, the open-set accuracy metric is computed through a post-hoc label-to-class mapping derived from the test ground truth, so the headline zero-shot numbers are not independent predictions, and the detection stage is never evaluated quantitatively. As a result, the central claim of a robust zero-shot ATR system is not supported by the evidence as presented.

major comments (4)
  1. [III-B] The open-set recognition accuracy is computed by 'selecting the most recurring keyword from the reevaluation labels corresponding to each ground truth class.' This is a post-hoc mapping from the model's free-text outputs to ground-truth classes, inferred from the test set after the predictions are made. Consequently, the open-set accuracies in Tables I-IV are not zero-shot evaluations: a model that consistently emits a plausible but different label (e.g., 'boat' for 'tank') would be scored as correct, and the numbers are not comparable to standard fixed-vocabulary or exact-match metrics. Because the open-set results are the primary evidence for the paper's claim of zero-shot recognition of novel classes, this evaluation protocol must be redone with a mapping fixed in advance (e.g., a predefined synonym dictionary that does not use test labels) or replaced by a proper open-set recognition metric (e.g., exact match or semantic similarity with a fixed threshold).
  2. [V and IV-A] The paper reports only recognition accuracy on the crops produced by the detector; it never reports detection precision, recall, missed-detection counts, or end-to-end system accuracy. Since the pipeline's overall ATR performance is bounded by the detection stage, the claim that the pipeline is 'a robust system for zero-shot ATR' (Abstract) is not quantitatively supported. Please provide per-dataset detection recall (and precision) for the binary 'vehicle' detection, and ideally report end-to-end recognition accuracy computed over all ground-truth objects rather than only over detected crops.
  3. [III-A] The statement that binary detection with the keyword 'vehicle' 'produced the same level of localization performance as keyword detection' is supported only by qualitative examples in Fig. 4. A quantitative comparison (e.g., IoU or recall at a fixed confidence threshold) is needed to justify this design choice and to assess how the detector prompt affects downstream recognition. Without such measurements, the effect of the detection stage on the reported system performance is unknown.
  4. [Tables I-IV] The numerical results are presented without any measure of variance, number of repeated trials, or statistical significance. Because LVLM outputs are stochastic (especially for API models) and the open-set metric is sensitive to the choice of the keyword mapping, the authors should report means and standard deviations over multiple runs or at least describe the number of trials and the exact scoring procedure.
minor comments (5)
  1. [III-B] The text contains a garbled sentence: 'Furthermore, for CoT recognition we use the following prompt of,For the closed-set recognition, we use the prompt,' which appears to be a duplication or editing error and should be corrected.
  2. [Table III] The header for the 3000-5000 m range lists 'Chain-of-Thought Open-set Closed-set' twice, making the column alignment unclear and the table hard to interpret.
  3. [Throughout] The model name 'LLaV A' appears throughout; it should be 'LLaVA'.
  4. [IV] The exact number of test samples per dataset and condition, and the complete verbatim prompts for all three recognition settings, are not specified; please include them for reproducibility.
  5. [References] Reference [30] is cited as a CLIP reference but is actually a Nature Reviews Methods Primers article; the CLIP paper is [29], so the citation appears misdirected.

Circularity Check

1 steps flagged · score 6.0 of 10

Open-set accuracy is computed via a post-hoc keyword-to-class mapping fitted to the test outputs, so the headline zero-shot numbers are not independent predictions.

  1. fitted input called prediction [Section III-B, Reevaluation phase; results in Tables I-IV]
    "Upon evaluating the results, we found that the labels generated during the reevaluation phase often differed from the ground truth labels. To address this discrepancy and reconcile the model’s output with the ground truth, we adopted a strategy of selecting the most recurring keyword from the reevaluation labels corresponding to each ground truth class."

    The open-set accuracy is not computed with a fixed, pre-specified label vocabulary. Instead, the paper first observes the LVLM's free-text outputs for each ground-truth class, then selects the most frequently produced keyword as the accepted label for that class. This mapping is derived from the test set after the fact, so the reported open-set accuracies in Tables I-IV are not independent zero-shot evaluations: for each class, the metric picks the label pattern the model actually emitted most often, regardless of whether that pattern is semantically correct. A model that consistently outputs a wrong but self-consistent keyword is scored as correct by construction.

full rationale

The pipeline itself is a straightforward empirical integration of an open-vocabulary detector with LVLMs, and its localization behavior, false-positive removal, and closed-set protocols are not circular: the closed-set and chain-of-thought closed-set evaluations provide a fixed label list and are less affected by post-hoc alignment. The single significant circularity is the open-set scoring protocol in Section III-B, which fits the evaluation vocabulary to the test outputs by selecting the most recurring keyword per ground-truth class. Because that protocol underlies the headline zero-shot open-set claims, the score is 6 rather than 0. No load-bearing self-citation chain appears: reference [43] is a prior open-set ATR paper by overlapping authors, but it is used only to supply the term 'open-set recognition' and does not itself justify the reported results.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The central claim rests on the effectiveness of the detection stage, the zero-shot validity of the LVLM, and the validity of the post-hoc label mapping used in evaluation. No new physical or model entities are introduced.

free parameters (1)
  • class-to-keyword alignment mapping = most recurring keyword per ground truth class (dataset-dependent)
    Used to compute open-set recognition accuracy; determined post hoc from the test outputs and ground truth, not predefined.
assumptions (3)
  • domain assumption YOLO-world's binary detection with prompt 'vehicle' localizes all relevant targets in the scene at acceptable precision
    The pipeline's detection stage relies on this; no detection metrics are reported to verify.
  • domain assumption The LVLM training data does not include the specific test images or exact vehicle classes, so performance is genuinely zero-shot
    The paper claims zero-shot for novel classes, but for common classes like tank and truck, LVLMs may have seen similar images during pretraining.
  • domain assumption Cropped vehicle images preserve sufficient context for LVLM recognition
    The pipeline feeds crops to LVLMs; if crops are too tight or noisy, recognition fails.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Zero-Shot Scene Understanding for Automatic Target Recognition Using Large Vision-Language Models." pith.science (2026). https://pith.science/paper/MXXLFBRB

@misc{pith2026250107396,
  author       = {Pith},
  title        = {Pith review of: Zero-Shot Scene Understanding for Automatic Target Recognition Using Large Vision-Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MXXLFBRB}},
  note         = {Machine review of arXiv:2501.07396}
}
read the original abstract

Automatic target recognition (ATR) plays a critical role in tasks such as navigation and surveillance, where safety and accuracy are paramount. In extreme use cases, such as military applications, these factors are often challenged due to the presence of unknown terrains, environmental conditions, and novel object categories. Current object detectors, including open-world detectors, lack the ability to confidently recognize novel objects or operate in unknown environments, as they have not been exposed to these new conditions. However, Large Vision-Language Models (LVLMs) exhibit emergent properties that enable them to recognize objects in varying conditions in a zero-shot manner. Despite this, LVLMs struggle to localize objects effectively within a scene. To address these limitations, we propose a novel pipeline that combines the detection capabilities of open-world detectors with the recognition confidence of LVLMs, creating a robust system for zero-shot ATR of novel classes and unknown domains. In this study, we compare the performance of various LVLMs for recognizing military vehicles, which are often underrepresented in training datasets. Additionally, we examine the impact of factors such as distance range, modality, and prompting methods on the recognition performance, providing insights into the development of more reliable ATR systems for novel conditions and classes.

Figures

Figures reproduced from arXiv: 2501.07396 by the authors.

Figure 1
Figure 1. Comparison between existing architectures zero-shot text prompted [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The proposed for ATR using LVLMs. First, in the ‘Detection phase,’ [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Sample images from the datasets depicting differences between [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Misrecognition by open-world detectors for novel object categories [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: The pipeline can be used to remove false positives (left image) [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Qualitative examples for ATR under weather degradation. The [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

55 extracted references · 32 canonical work pages

  1. [1]

    Automatic target recognition: State of the art survey,

    B. Bhanu, “Automatic target recognition: State of the art survey,” IEEE transactions on aerospace and electronic systems , no. 4, pp. 364–379, 1986

    work page 1986

  2. [2]

    The automatic target-recognition system in saip,

    L. M. Novak, G. J. Owirka, W. S. Brower, and A. L. Weaver, “The automatic target-recognition system in saip,” Lincoln Laboratory Journal, vol. 10, no. 2, 1997

    work page 1997

  3. [3]

    Automatic target recognition based on simultaneous sparse representation,

    V . M. Patel, N. M. Nasrabadi, and R. Chellappa, “Automatic target recognition based on simultaneous sparse representation,” in 2010 IEEE international conference on image processing . IEEE, 2010, pp. 1377–1380

    work page 2010

  4. [4]

    Distilling the knowledge in a neural network,

    G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,” arXiv preprint arXiv:1503.02531 , 2015

    arXiv 2015

  5. [5]

    Accelerating very deep convo- lutional networks for classification and detection,

    X. Zhang, J. Zou, K. He, and J. Sun, “Accelerating very deep convo- lutional networks for classification and detection,” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 38, no. 10, pp. 1943–1955, 2015

    work page 1943

  6. [6]

    Object recognition and detection with deep learning for autonomous driving applications,

    A. Uc ¸ar, Y . Demir, and C. G¨uzelis ¸, “Object recognition and detection with deep learning for autonomous driving applications,” Simulation, vol. 93, no. 9, pp. 759–769, 2017

    work page 2017

  7. [7]

    Review of current aided/automatic target acquisition technology for military target acquisition tasks,

    J. A. Ratches, “Review of current aided/automatic target acquisition technology for military target acquisition tasks,” Optical Engineering, vol. 50, no. 7, pp. 072 001–072 001, 2011

    work page 2011

  8. [8]

    Ar- tificial intelligence for national security: the predictability problem,

    M. Taddeo, M. Ziosi, A. Tsamados, L. Gilli, and S. Kurapati, “Ar- tificial intelligence for national security: the predictability problem,” Centre for Digital Ethics (CEDE) Research Paper No. F orthcoming , 2022

    work page 2022

Show all 55 references
  1. [9]

    Autonomous vehicles and intelligent automation: Applications, challenges, and opportuni- ties,

    G. Bathla, K. Bhadane, R. K. Singh, R. Kumar, R. Aluvalu, R. Krish- namurthi, A. Kumar, R. Thakur, and S. Basheer, “Autonomous vehicles and intelligent automation: Applications, challenges, and opportuni- ties,” Mobile Information Systems , vol. 2022, no. 1, p. 7632892, 2022

    2022

  2. [10]

    Concrete problems in ai safety,

    D. Amodei, C. Olah, J. Steinhardt, P. Christiano, J. Schulman, and D. Man ´e, “Concrete problems in ai safety,” arXiv preprint arXiv:1606.06565, 2016

    2016 arXiv

  3. [11]

    Unsolved problems in ML safety,

    D. Hendrycks, N. Carlini, J. Schulman, and J. Steinhardt, “Unsolved problems in ML safety,” arXiv preprint arXiv:2109.13916 , 2021

    2021 arXiv

  4. [12]

    Generalized out-of-distribution detection: A survey,

    J. Yang, K. Zhou, Y . Li, and Z. Liu, “Generalized out-of-distribution detection: A survey,” International Journal of Computer Vision , pp. 1–28, 2024

    2024

  5. [13]

    Generalized out-of-distribution detection and beyond in vision language model era: A survey,

    A. Miyai, J. Yang, J. Zhang, Y . Ming, Y . Lin, Q. Yu, G. Irie, S. Joty, Y . Li, H. Li et al. , “Generalized out-of-distribution detection and beyond in vision language model era: A survey,” arXiv preprint arXiv:2407.21794, 2024

    2024 arXiv

  6. [14]

    Meta-uda: Unsupervised domain adaptive thermal object detection using meta- learning,

    V . Vs, D. Poster, S. You, S. Hu, and V . M. Patel, “Meta-uda: Unsupervised domain adaptive thermal object detection using meta- learning,” in proceedings of the IEEE/CVF winter conference on applications of computer vision , 2022, pp. 1412–1423

    2022

  7. [15]

    On the validity of bayesian neural net- works for uncertainty estimation,

    J. Mitros and B. Mac Namee, “On the validity of bayesian neural net- works for uncertainty estimation,” arXiv preprint arXiv:1912.01530 , 2019

    1912 arXiv

  8. [16]

    Knowing the unknown: Open-world recognition for biodiversity datasets,

    R. Gangireddy, “Knowing the unknown: Open-world recognition for biodiversity datasets,” Master’s thesis, University of Twente, 2023

    2023

  9. [17]

    The malicious use of artificial intelligence: Forecasting, prevention, and mitigation,

    M. Brundage, S. Avin, J. Clark, H. Toner, P. Eckersley, B. Garfinkel, A. Dafoe, P. Scharre, T. Zeitzoff, B. Filar et al., “The malicious use of artificial intelligence: Forecasting, prevention, and mitigation,” arXiv preprint arXiv:1802.07228, 2018

    2018 arXiv

  10. [18]

    The impact of cooperative perception on decision making and planning of autonomous vehicles,

    S.-W. Kim, W. Liu, M. H. Ang, E. Frazzoli, and D. Rus, “The impact of cooperative perception on decision making and planning of autonomous vehicles,” IEEE Intelligent Transportation Systems Magazine, vol. 7, no. 3, pp. 39–50, 2015

    2015

  11. [19]

    Towards open world object detection,

    K. Joseph, S. Khan, F. S. Khan, and V . N. Balasubramanian, “Towards open world object detection,” in Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , 2021, pp. 5830– 5840

    2021

  12. [20]

    Unidentified video objects: A benchmark for dense, open-world segmentation,

    W. Wang, M. Feiszli, H. Wang, and D. Tran, “Unidentified video objects: A benchmark for dense, open-world segmentation,” in Pro- ceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 10 776–10 785

    2021

  13. [21]

    Breaking the closed world assumption in text classification,

    G. Fei and B. Liu, “Breaking the closed world assumption in text classification,” in Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , 2016, pp. 506–514

    2016

  14. [22]

    Dynamic few-shot visual learning with- out forgetting,

    S. Gidaris and N. Komodakis, “Dynamic few-shot visual learning with- out forgetting,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2018, pp. 4367–4375

    2018

  15. [23]

    Online incremental learning algorithm for anomaly detection and prediction in health care,

    K. Raghuraman, M. Senthurpandian, M. Shanmugasundaram, V . Vaidehiet al., “Online incremental learning algorithm for anomaly detection and prediction in health care,” in 2014 International Con- ference on Recent Trends in Information Technology . IEEE, 2014, pp. 1–6

    2014

  16. [24]

    Detecting everything in the open world: Towards universal object detection,

    Z. Wang, Y . Li, X. Chen, S.-N. Lim, A. Torralba, H. Zhao, and S. Wang, “Detecting everything in the open world: Towards universal object detection,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 11 433–11 443

    2023

  17. [25]

    Lifelong machine learning: a paradigm for continuous learn- ing,

    B. Liu, “Lifelong machine learning: a paradigm for continuous learn- ing,” Frontiers of Computer Science, vol. 11, no. 3, pp. 359–361, 2017

    2017

  18. [26]

    Advancing autonomy through lifelong learning: a survey of autonomous intelligent systems,

    D. Zhu, Q. Bu, Z. Zhu, Y . Zhang, and Z. Wang, “Advancing autonomy through lifelong learning: a survey of autonomous intelligent systems,” Frontiers in Neurorobotics, vol. 18, p. 1385778, 2024

    2024

  19. [27]

    Vision-language models for vision tasks: A survey,

    J. Zhang, J. Huang, S. Jin, and S. Lu, “Vision-language models for vision tasks: A survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

    2024

  20. [28]

    A survey of vision-language pre-trained models,

    Y . Du, Z. Liu, J. Li, and W. X. Zhao, “A survey of vision-language pre-trained models,” arXiv preprint arXiv:2202.10936 , 2022

    2022 arXiv

  21. [29]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark et al., “Learning transferable visual models from natural language supervision,” in ICML, 2021

    2021

  22. [30]

    Clip and complementary meth- ods,

    M. Hafner, M. Katsantoni, T. K ¨oster, J. Marks, J. Mukherjee, D. Staiger, J. Ule, and M. Zavolan, “Clip and complementary meth- ods,” Nature Reviews Methods Primers , vol. 1, no. 1, pp. 1–23, 2021

    2021

  23. [31]

    On the safety concerns of deploying llms/vlms in robotics: Highlighting the risks and vulnerabilities,

    X. Wu, R. Xian, T. Guan, J. Liang, S. Chakraborty, F. Liu, B. Sadler, D. Manocha, and A. S. Bedi, “On the safety concerns of deploying llms/vlms in robotics: Highlighting the risks and vulnerabilities,” arXiv preprint arXiv:2402.10340, 2024

    2024 arXiv

  24. [32]

    Applications of large language models for robot navigation and scene understanding,

    W. Chen, “Applications of large language models for robot navigation and scene understanding,” Ph.D. dissertation, Massachusetts Institute of Technology, 2023

    2023

  25. [33]

    Pivot: Iterative visual prompting elicits actionable knowledge for vlms,

    S. Nasiriany, F. Xia, W. Yu, T. Xiao, J. Liang, I. Dasgupta, A. Xie, D. Driess, A. Wahid, Z. Xu et al. , “Pivot: Iterative visual prompting elicits actionable knowledge for vlms,” arXiv preprint arXiv:2402.07872, 2024

    2024 arXiv

  26. [34]

    Yolo-world: Real-time open-vocabulary object detection,

    T. Cheng, L. Song, Y . Ge, W. Liu, X. Wang, and Y . Shan, “Yolo-world: Real-time open-vocabulary object detection,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 16 901–16 911

    2024

  27. [35]

    Towards open world recognition,

    A. Bendale and T. Boult, “Towards open world recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 1893–1902

    2015

  28. [36]

    Ow-detr: Open-world detection transformer,

    A. Gupta, S. Narayan, K. Joseph, S. Khan, F. S. Khan, and M. Shah, “Ow-detr: Open-world detection transformer,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2022, pp. 9235–9244

    2022

  29. [37]

    Exploring vision-language foundation model for novel object captioning,

    J. Luo, Y . Li, Y . Pan, T. Yao, J. Feng, H. Chao, and T. Mei, “Exploring vision-language foundation model for novel object captioning,” IEEE Transactions on Circuits and Systems for Video Technology , 2024

    2024

  30. [38]

    Improved open world object detection using class-wise feature space learning,

    M. A. Iqbal, Y . C. Yoon, M. U. Khan, and S. K. Kim, “Improved open world object detection using class-wise feature space learning,” IEEE Access, vol. 11, pp. 131 221–131 236, 2023

    2023

  31. [39]

    Self-supervised features improve open-world learning,

    A. R. Dhamija, T. Ahmad, J. Schwan, M. Jafarzadeh, C. Li, and T. E. Boult, “Self-supervised features improve open-world learning,” arXiv preprint arXiv:2102.07848, 2021

    2021 arXiv

  32. [40]

    Can foundation models wrangle your data?

    A. Narayan, I. Chami, L. Orr, S. Arora, and C. R ´e, “Can foundation models wrangle your data?” arXiv preprint arXiv:2205.09911 , 2022

    2022 arXiv

  33. [41]

    Segment anything,

    A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y . Lo et al. , “Segment anything,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2023, pp. 4015–4026

    2023

  34. [42]

    Dremel: interactive analysis of web- scale datasets,

    S. Melnik, A. Gubarev, J. J. Long, G. Romer, S. Shivakumar, M. Tolton, and T. Vassilakis, “Dremel: interactive analysis of web- scale datasets,” Proceedings of the VLDB Endowment , vol. 3, no. 1-2, pp. 330–339, 2010

    2010

  35. [43]

    Open-set automatic target recognition,

    B. Safaei, V . Vibashan, C. M. de Melo, S. Hu, and V . M. Patel, “Open-set automatic target recognition,” in ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2023, pp. 1–5

    2023

  36. [44]

    Hello gpt-4o,

    OpenAI, “Hello gpt-4o,” https://openai.com/index/hello-gpt-4o/. [Online]. Available: https://openai.com/index/hello-gpt-4o/

  37. [45]

    Introducing the next generation of Claude,

    “Introducing the next generation of Claude,” https://www.anthropic.com/news/claude-3-family. [Online]. Available: https://www.anthropic.com/news/claude-3-family 7

  38. [46]

    Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context,

    M. Reid, N. Savinov, D. Teplyashin, D. Lepikhin, T. Lillicrap, J.-b. Alayrac, R. Soricut, A. Lazaridou, O. Firat, J. Schrittwieser et al. , “Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context,” arXiv preprint arXiv:2403.05530 , 2024

    2024 arXiv

  39. [47]

    Visual instruction tuning,

    H. Liu, C. Li, Q. Wu, and Y . J. Lee, “Visual instruction tuning,” 2023

    2023

  40. [48]

    Phi- 3 technical report: A highly capable language model locally on your phone,

    M. Abdin, S. A. Jacobs, A. A. Awan, J. Aneja, A. Awadallah, H. Awadalla, N. Bach, A. Bahree, A. Bakhtiari, H. Behl et al. , “Phi- 3 technical report: A highly capable language model locally on your phone,” arXiv preprint arXiv:2404.14219 , 2024

    2024 arXiv

  41. [49]

    Minicpm-v: A gpt-4v level mllm on your phone,

    Y . Yao, T. Yu, A. Zhang, C. Wang, J. Cui, H. Zhu, T. Cai, H. Li, W. Zhao, Z. He et al. , “Minicpm-v: A gpt-4v level mllm on your phone,” arXiv preprint arXiv:2408.01800 , 2024

    2024 arXiv

  42. [50]

    Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks,

    Z. Chen, J. Wu, W. Wang, W. Su, G. Chen, S. Xing, M. Zhong, Q. Zhang, X. Zhu, L. Lu, B. Li, P. Luo, T. Lu, Y . Qiao, and J. Dai, “Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks,” arXiv preprint arXiv:2312.14238 , 2023

    2023 arXiv

  43. [51]

    Llava-next: Improved reasoning, ocr, and world knowledge,

    H. Liu, C. Li, Y . Li, B. Li, Y . Zhang, S. Shen, and Y . J. Lee, “Llava-next: Improved reasoning, ocr, and world knowledge,” January 2024. [Online]. Available: https://llava-vl.github.io/blog/ 2024-01-30-llava-next/

    2024

  44. [52]

    Cogvlm: Visual expert for pretrained language models,

    W. Wang, Q. Lv, W. Yu, W. Hong, J. Qi, Y . Wang, J. Ji, Z. Yang, L. Zhao, X. Song et al. , “Cogvlm: Visual expert for pretrained language models,” arXiv preprint arXiv:2311.03079 , 2023

    2023 arXiv

  45. [53]

    Openflamingo: An open-source framework for training large autoregressive vision- language models,

    A. Awadalla, I. Gao, J. Gardner, J. Hessel, Y . Hanafy, W. Zhu, K. Marathe, Y . Bitton, S. Gadre, S. Sagawa, J. Jitsev, S. Kornblith, P. W. Koh, G. Ilharco, M. Wortsman, and L. Schmidt, “Openflamingo: An open-source framework for training large autoregressive vision- language ...

    2023 arXiv

  46. [54]

    Instructblip: Towards general-purpose vision- language models with instruction tuning,

    W. Dai, J. Li, D. Li, A. M. H. Tiong, J. Zhao, W. Wang, B. Li, P. Fung, and S. Hoi, “Instructblip: Towards general-purpose vision- language models with instruction tuning,” 2023

    2023

  47. [55]

    Blip-2: Bootstrapping language- image pre-training with frozen image encoders and large language models,

    J. Li, D. Li, S. Savarese, and S. Hoi, “Blip-2: Bootstrapping language- image pre-training with frozen image encoders and large language models,” in International conference on machine learning . PMLR, 2023, pp. 19 730–19 742. 8

    2023

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.