OW-CLIP: Data-Efficient Visual Supervision for Open-World Object Detection via Human-AI Collaboration

Jiazhi Xia; Junwen Duan; Shixia Liu; Wei Xue; Ziyao Kang

REVIEW 3 major objections 5 minor 63 references

OW-CLIP: Data-Efficient Visual Supervision for Open-World Object Detection via Human-AI Collaboration

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A CLIP-based detector reaches 89 percent of state-of-the-art open-world detection performance using only 3.8 percent of the training data.

desk verdict The 89%/3.8% data-efficiency claim is unverifiable until the authors disclose which split the curated annotations came from, but the system and degradation study are worth taking seriously and the paper merits peer review. read the letter →

arxiv 2507.19870 v1 pith:PIZOVLYX submitted 2025-07-26 cs.CV cs.HC

Junwen Duan , Wei Xue , Ziyao Kang , Shixia Liu , Jiazhi Xia This is my paper · ORCID

classification cs.CVcs.HC

keywords open-worldobjectdetectionCLIPprompttuningdata-efficientsupervisionhuman-AIcollaborationCrop-Smoothingvisualanalyticsannotationquality

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to show that the obstacle to practical open-world object detection is not model architecture but the quality and cost of supervision. It proposes a human-AI collaboration system in which a large language model proposes visual feature phrases for a newly discovered class, a CLIP similarity score filters candidate images, and a user marks images as simple or hard. Training then uses prompt tuning with a Crop-Smoothing objective that scales confidence with object completeness. The central quantitative claim is that this recipe reaches about 89 percent of state-of-the-art performance on the second task while using only 3.8 percent of the annotations, and beats the state of the art when both are given the same data volume. If true, open-world detectors can be extended to new classes in minutes of annotation rather than large crowdsourced labeling campaigns.

What carries the argument

The mechanism that carries the argument is a human-in-the-loop data-curation loop wrapped around frozen CLIP encoders. For a newly named class, an LLM generates candidate visual feature phrases, which a user filters down to a few; the selected phrases are encoded and averaged to initialize a learnable class text vector. Candidate images are scored by CLIP image-text similarity and split into Simple (complete object) and Hard (partial or cluttered) sets, with an interactive density plot guiding threshold choices. During training, Crop-Smoothing randomly crops each image and assigns the ground-truth label a confidence that decreases with the cropping ratio, spreading the leftover probability over other classes, which stops the model from treating local patches as sufficient evidence. Prompt tuning then appends learnable tokens to each ViT layer of the image encoder and per-class context vectors, leaving all pretrained parameters frozen, so each new class is added as a separate plug-in module.

What would settle it

Reproduce task 2 with curated images drawn strictly from the official training split and evaluated on the official test split; if the 89-percent-of-SOTA figure drops materially under this strict split, the claim depends on evaluation leakage.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that a small amount of highly curated dual-modal supervision can stand in for most of the crowdsourced labels that current open-world object detection methods require. In the standard Pascal VOC/MS-COCO open-world setup, the system reports 53.2 mAP in task 1 and 40.0 mAP in task 2, reaching 89 percent of the state-of-the-art result in task 2 while using only 3.8 percent of the annotations, and it outperforms the state of the art when both use the same data volume. The authors attribute the result to three interacting components: class-specific feature phrases that initialize text prompts, a Crop-Smoothing objective that ties classification confidence to object completeness, and human filtering that separates images into Simple and Hard sets. Ablation results are presented as evidence that each component contributes, with Crop-Smoothing and image differentiation producing the largest performance drops when removed.

Load-bearing premise

The comparison assumes that the images users curated, and the Delete/Reserve filtering they applied, did not remove difficult cases from the evaluation split, since the paper does not state which image split its curated annotations came from.

Editorial extensions

If this is right

A new class can be added with roughly 220 curated images and four selected feature phrases, taking about 4 minutes for simple classes and 6-9 minutes for more complex ones.
Because the proposal generator and pretrained parameters stay frozen, the same detector can be extended to new classes without retraining old ones or changing the architecture.
The forgetting rate for previously learned classes stays at 8 percent after adding 20 classes, compared with 12 percent for the SOTA method, so incremental updates are less destructive.
Removing Crop-Smoothing costs about 9.8 mAP points in task 1 and 6.7 points in task 2, making the completeness-aware confidence the largest single contributor among the system's components.
Classes learned in different episodes are not trained to be distinguished from each other, so users must group confusable classes in the same episode to avoid confusion.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A natural extension beyond the paper is to automate the Simple/Hard split with an object-completeness estimator, since the ablation attributes most of the gain to the human image-differentiation step; if an automatic proxy works, the pipeline becomes nearly annotation-free.
Because image filtering is driven by CLIP similarity, the system inherits CLIP's embedding biases; classes that are visually distant from CLIP's pretraining vocabulary may yield poor candidate sets, so the 3.8-percent data claim is most likely to generalize for classes close to CLIP's distribution.
The paper's own limitation discussion notes that LLM phrases can be redundant or misleading, such as emphasizing striped texture for zebra; a testable fix is to add a phrase-diversity penalty or rank phrases by mutual information before presenting them to the user.
The equal-data comparison with the state of the art uses randomly sampled images for the baseline; a further stress test would compare against an active-learning baseline that also selects hard examples, isolating whether the gain comes from curation or from the Crop-Smoothing objective.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague, here's my take on OW-CLIP. The headline claim—89% of SOTA at 3.8% of the data—is the thing to scrutinize. The paper never says which split of the OWOD dataset the curated annotations were pulled from. If they came from the same images used for evaluation, the comparison in Table 1 is not a data-efficiency result; it's test-set contamination. The sentence in Section 6.1 saying the annotations 'differ from the original dataset's train instances' actually raises the alarm rather than quelling it, because it rules out the train split without saying what the alternative was. This is the load-bearing issue, and it makes the central quantitative claim unverifiable as written.

That's the soft spot in proportion. The rest of the paper is better than average for a system paper. The integration is genuinely new: human-in-the-loop annotation with LLM-generated visual phrases, CLIP prompt tuning, and the Crop-Smoothing trick. Crop-Smoothing is a reparameterization of label smoothing on random crops, but applying it to open-world incremental training is a legitimate idea, and the degradation study (Table 3) gives real internal support—removing it causes a large drop. The comparison to OW-Adapter is the right kind of direct comparison. The limitations section honestly names episode-level class confusion and phrase misguidance, which is more than many papers do.

The other concerns are real but secondary: no code or curated data released, single-run numbers, annotation counts and thresholds tuned during the experiment, and the 'outperforms SOTA with equivalent data' line overstates because RandBox-S is a random subsample, not the SOTA method itself. If the split issue were resolved, I'd call this a solid system paper with a credible data-efficiency story. As it stands, the central number is unproven.

Who is this for? People working on open-world detection, human-in-the-loop data curation, and visualization-assisted annotation. It deserves a serious referee—the idea is important enough and the system is substantial enough that an editor should send it out, with a strong request to state the split, release the artifacts, and add error bars. If the split is clean, it's a conditional accept; if not, it's a reject for leakage. I'd bring this to reading group for the discussion, but I wouldn't cite the quantitative claim in its current form.

Referee Report

3 major / 5 minor

Summary. The paper presents OW-CLIP, a visual analytics system for open-world object detection that combines LLM-generated visual feature phrases, interactive image curation into Simple and Hard categories, and multimodal prompt tuning with a novel Crop-Smoothing technique. The central claim is that this human-AI collaboration pipeline lets a frozen-CLIP detector reach 89% of state-of-the-art mAP while using only 3.8% of the training annotations, and that it outperforms RandBox when trained with equivalent data volumes. The evaluation includes a benchmark comparison (Table 1), a comparison with OW-Adapter (Table 2), a degradation study (Table 3), a case study, and a user study.

Significance. If the data-efficiency result is valid, OW-CLIP would be a meaningful step toward practical open-world detection: it offers a plug-and-play prompt-tuning alternative to architectural modifications, and the degradation study in Table 3 gives component-level evidence that Crop-Smoothing, phrase selection, Simple/Hard differentiation, and LLM phrase generation each contribute to performance. The paper also makes concrete, falsifiable claims about annotation effort and usability through the case study and user study. However, the headline 89%/3.8% result rests on data-provenance details that are not currently documented, so the significance cannot be fully assessed without clarification.

major comments (3)

[Section 6.1, Dataset paragraph] The paper states that images from the conventional OWOD dataset were processed and annotated with the visualization system and that the annotated results 'differ from the original dataset’s train instances,' but it never states which split (train/val/test) these images were drawn from. In the standard OWOD protocol, comparison methods are trained on official train splits and evaluated on held-out images; if the curated set includes evaluation images, the 89%/3.8% claim in the Abstract and Conclusion would be inflated by test-set leakage. Please state the exact split, verify disjointness from the evaluation set, and release the curated data or a detailed split description so the comparison can be audited.
[Table 1 and Section 6.1, Result (4)] The reported 3.8% data-efficiency figure depends on the annotation quantity and on the filtering thresholds (ls, hs, lh, hh in Section 5.2.2) being tuned to maximize validation mAP, as shown by the statement that 'optimal performance was achieved with around 220 images and four feature phrases per class.' Because all experiments are single-run, the reader cannot tell whether the 89% number is robust to run-to-run variation or to modest changes in these thresholds. Please provide multiple seeds with mean and standard deviation, and a sensitivity analysis in which annotation count and thresholds are varied.
[Table 1, RandBox-S row] The comparison claims that OW-CLIP 'outperforms SOTA approach when trained with equivalent data volumes,' but RandBox-S is described only as a randomly sampled subset of the original RandBox dataset, with no specification of how the subset was constructed (e.g., which images were selected, how the balanced class distribution was enforced, or how many runs were averaged). Since RandBox-S is trained on random images while OW-CLIP uses curated and filtered images, the row conflates data volume with annotation effort; more detail on the sampling protocol is needed for the equivalence claim to be verifiable.

minor comments (5)

[Throughout] The manuscript contains several typos and grammatical errors, including 'develope' and 'quaily' in the Abstract, 'traning' in Section 6.1, 'multimodel' in Implementation Details, 'visulization' in Section 5.1, and 'Threhold' in Figure 1.
[Figure 11] The example text in Figure 11 appears truncated or misformatted ('Zebra? lBlack and white striped pattern'), and the figure caption does not explain what the model predicted instead of the correct class. Please clean the figure and add a short explanation of the misclassification.
[Table 1] The caption does not define what the Faster-RCNN row represents: is it the standard detector trained on the same known classes, and was it retrained after each task? Please clarify the training protocol for this baseline.
[Section 6.1, Implementation Details] Please report the optimizer settings, prompt-learning rates, and any weight decay or scheduling used for the multimodal prompt tuning, since these details are important for reproducibility.
[Section 5.2.2] The definitions of Simple and Hard images are qualitative; please provide the exact threshold ranges used in the experiments so that other researchers can replicate the curation process.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity; the 89%/3.8% result is an external benchmark comparison, with only minor non-load-bearing self-citations.

full rationale

The paper's central quantitative claim is an external benchmark comparison rather than a derivation: Table 1 reports mAP on the standard Pascal VOC / MS-COCO OWOD benchmark against baselines such as RandBox, and the 3.8% figure is the ratio of OW-CLIP's annotation count to the baseline's annotation count. The Crop-Smoothing loss (Eq. 1) and multimodal prompt tuning are training mechanisms, not predictions; neither reduces to the evaluation metric by construction. The curation pipeline does use the frozen CLIP encoder to compute similarity (Eq. 3) for image filtering and later prompt-tunes the same CLIP backbone, which is a self-referential design choice and a potential confound, but it does not make the reported test-set mAP equivalent to the training filter by construction. The self-citations in the paper (e.g., [52], [12], [53], [32]) support visualization design decisions such as t-SNE-based cluster display and interactive correction; they are not load-bearing for the headline result and no uniqueness theorem is imported from the authors' own prior work. The most serious caveat is external validity, not circularity: Section 6.1 says the curated annotations were made from images in the conventional OWOD dataset that 'differ from the original dataset's train instances' without naming the split, so a test-set leakage risk exists; that is a benchmark-integrity concern that cannot be verified from the paper and does not constitute a circular derivation chain.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The method adds no new physical or architectural entities with independent falsifiable handles. Its empirical claims rest on five hand-selected quantities and on domain assumptions about completeness confidence, CLIP similarity as relevance, fairness of the undisclosed data split, and the value of human judgments. The free parameters and assumptions determine the size and composition of the training set, so they directly shape the reported 89 percent figure.

free parameters (5)

Classification confidence threshold t = Not reported
Proposals with similarity above t are classified known; those below t are classified unknown. No value or sensitivity analysis is given.
Vision prompt length M = 10
Set to 10 for all experiments; no ablation shows sensitivity to this choice.
Crop-Smoothing confidence D_epsilon = 1.0 for Simple images; unspecified for Hard images
The training target depends on the crop ratio epsilon through D_epsilon, but the full mapping and epsilon_min are not specified, leaving the label-smoothing strength as a free design choice.
Image filtering thresholds ls, hs, lh, hh = Zebra case study: 0.3349 and 0.3522
These thresholds control which images enter the Simple and Hard training sets and are adjusted per class during annotation, so they are effectively fit to the data.
Annotation quantity per class = Approximately 220 images and 4 text phrases
The paper states optimal performance was achieved at this level, a post hoc selection made on the same evaluation benchmark.

assumptions (5)

domain assumption Classification confidence should increase monotonically with object completeness in the image
Section 4.1 founds Crop-Smoothing on this principle, supported only by one illustrative zebra example in Figure 3.
domain assumption Pre-trained CLIP image-text similarity is a valid proxy for image relevance to a class
Section 5.1 uses CLIP similarity to define Simple and Hard candidate sets and to filter training images.
domain assumption The curated training images are drawn from a distribution that allows fair comparison with conventional OWOD baselines
Section 6.1 states the annotated data differ from the original dataset's train instances without specifying the split, implicitly assuming no test-set leakage and comparable image difficulty.
domain assumption Freezing the proposal generator and adapting only the classifier is sufficient because classification bias is the core OWOD issue
Section 4 invokes prior studies [29, 47] for this claim; no experiment in the paper varies the proposal generator.
domain assumption Human Simple and Hard judgments and phrase selections improve model training
This is supported only by the paper's own ablation experiments, not by independent external evidence.

how reviews work

0 comments

Cite this review

Pith. "Pith review of OW-CLIP: Data-Efficient Visual Supervision for Open-World Object Detection via Human-AI Collaboration." pith.science (2026). https://pith.science/paper/PIZOVLYX

@misc{pith2026250719870,
  author       = {Pith},
  title        = {Pith review of: OW-CLIP: Data-Efficient Visual Supervision for Open-World Object Detection via Human-AI Collaboration},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PIZOVLYX}},
  note         = {Machine review of arXiv:2507.19870}
}

read the original abstract

Open-world object detection (OWOD) extends traditional object detection to identifying both known and unknown object, necessitating continuous model adaptation as new annotations emerge. Current approaches face significant limitations: 1) data-hungry training due to reliance on a large number of crowdsourced annotations, 2) susceptibility to "partial feature overfitting," and 3) limited flexibility due to required model architecture modifications. To tackle these issues, we present OW-CLIP, a visual analytics system that provides curated data and enables data-efficient OWOD model incremental training. OW-CLIP implements plug-and-play multimodal prompt tuning tailored for OWOD settings and introduces a novel "Crop-Smoothing" technique to mitigate partial feature overfitting. To meet the data requirements for the training methodology, we propose dual-modal data refinement methods that leverage large language models and cross-modal similarity for data generation and filtering. Simultaneously, we develope a visualization interface that enables users to explore and deliver high-quality annotations: including class-specific visual feature phrases and fine-grained differentiated images. Quantitative evaluation demonstrates that OW-CLIP achieves competitive performance at 89% of state-of-the-art performance while requiring only 3.8% self-generated data, while outperforming SOTA approach when trained with equivalent data volumes. A case study shows the effectiveness of the developed method and the improved annotation quality of our visualization system.

Figures

Figures reproduced from arXiv: 2507.19870 by the authors.

**Figure 3.** Visualization of partial feature overfitting. The classification [PITH_FULL_IMAGE:figures/full_fig_p002_3.png] view at source ↗

**Figure 4.** Given an image of a complete object, we randomly crop it to gener [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗

Figures from the paper (5 more)

**Figure 7.** Figure 7: Density distribution of Simple (skyblue) and Hard (salmon) images [PITH_FULL_IMAGE:figures/full_fig_p005_7.png]

**Figure 8.** Figure 8: Annotation interface for OW-CLIP. (a) Cluster scatter plot of all unknown detection proposals. User can perform lasso selection on the point [PITH_FULL_IMAGE:figures/full_fig_p006_8.png]

**Figure 9.** Figure 9: Cluster Recommendation. The lasso tool was used to inspect [PITH_FULL_IMAGE:figures/full_fig_p006_9.png]

**Figure 10.** Figure 10: Related-Image Recommendation. Click on a "tie" image that may [PITH_FULL_IMAGE:figures/full_fig_p007_10.png]

**Figure 11.** Figure 11: An example of misguidance caused by feature phrases. [PITH_FULL_IMAGE:figures/full_fig_p009_11.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

63 extracted references · 51 canonical work pages

[1]

Z. Bai, P. Wang, T. Xiao, T. He, Z. Han, Z. Zhang, and M. Z. Shou. Hallucination of multimodal large language models: A survey.arXiv preprint arXiv:2404.18930, 2024. 6

arXiv 2024
[2]

Bendale and T

A. Bendale and T. Boult. Towards open world recognition. InProceedings of the IEEE conference on computer vision and pattern recognition, pp. 1893–1902, 2015. 2

work page 1902
[3]

Birhane, A

A. Birhane, A. Kasirzadeh, D. Leslie, and S. Wachter. Science in the age of large language models.Nature Reviews Physics, 5(5):277–280, 2023. 5

work page 2023
[4]

Caron, P

M. Caron, P. Bojanowski, A. Joulin, and M. Douze. Deep clustering for unsupervised learning of visual features. InProceedings of the European conference on computer vision (ECCV), pp. 132–149, 2018. 6

work page 2018
[5]

C. Chen, J. Wu, X. Wang, S. Xiang, S.-H. Zhang, Q. Tang, and S. Liu. Towards better caption supervision for object detection.IEEE Transactions on Visualization and Computer Graphics, 28(4):1941–1954, 2021. 3

work page 1941
[6]

F. D. Davis. Perceived usefulness, perceived ease of use, and user ac- ceptance of information technology.MIS quarterly, pp. 319–340, 1989. 9

work page 1989
[7]

De Angelis, F

L. De Angelis, F. Baglivo, G. Arzilli, G. P. Privitera, P. Ferragina, A. E. Tozzi, and C. Rizzo. Chatgpt and the rise of large language models: the new ai-driven infodemic threat in public health.Frontiers in public health, 11:1166120, 2023. 5

work page 2023
[8]

De Vries, F

H. De Vries, F. Strub, S. Chandar, O. Pietquin, H. Larochelle, and A. Courville. Guesswhat?! visual object discovery through multi-modal dialogue. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5503–5512, 2017. 2

work page 2017

Show all 63 references

[9]

Everingham, S

M. Everingham, S. A. Eslami, L. Van Gool, C. K. Williams, J. Winn, and A. Zisserman. The pascal visual object classes challenge: A retrospective. International journal of computer vision, 111:98–136, 2015. 7

2015
[10]

Everingham, L

M. Everingham, L. Van Gool, C. K. Williams, J. Winn, and A. Zisserman. The pascal visual object classes (voc) challenge.International journal of computer vision, 88:303–338, 2010. 7

2010
[11]

R. Faster. Towards real-time object detection with region pro- posal networks.Advances in neural information processing systems, 9199(10.5555):2969239–2969250, 2015. 7, 8

2015
[12]

Y . Feng, X. Wang, B. Pan, K. K. Wong, Y . Ren, S. Liu, Z. Yan, Y . Ma, H. Qu, and W. Chen. Xnli: Explaining and diagnosing nli-based visual data analysis.IEEE Transactions on Visualization and Computer Graphics,
[13]

E. Fini, E. Sangineto, S. Lathuilière, Z. Zhong, M. Nabi, and E. Ricci. A unified objective for novel class discovery. InProceedings of the IEEE/CVF international conference on computer vision, pp. 9284–9292,
[14]

D. Guo, D. Yang, H. Zhang, J. Song, R. Zhang, R. Xu, Q. Zhu, S. Ma, P. Wang, X. Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025. 5

2025 arXiv
[15]

Gupta, S

A. Gupta, S. Narayan, K. Joseph, S. Khan, F. S. Khan, and M. Shah. Ow- detr: Open-world detection transformer. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 9235–9244,
[16]

Hamerly and C

G. Hamerly and C. Elkan. Learning the k in k-means.Advances in neural information processing systems, 16, 2003. 6

2003
[17]

S. G. Hart and L. E. Staveland. Development of nasa-tlx (task load index): Results of empirical and theoretical research. InAdvances in psychology, vol. 52, pp. 139–183. Elsevier, 1988. 9

1988
[18]

Huang, W

L. Huang, W. Yu, W. Ma, W. Zhong, Z. Feng, H. Wang, Q. Chen, W. Peng, X. Feng, B. Qin, et al. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions.ACM Transactions on Information Systems, 43(2):1–55, 2025. 6

2025
[19]

Jamonnak, J

S. Jamonnak, J. Guo, W. He, L. Gou, and L. Ren. Ow-adapter: Human- assisted open-world object detection with a few examples.IEEE Transac- tions on Visualization and Computer Graphics, 2023. 2, 3, 7

2023
[20]

M. Jia, L. Tang, B.-C. Chen, C. Cardie, S. Belongie, B. Hariharan, and S.-N. Lim. Visual prompt tuning. InEuropean Conference on Computer Vision, pp. 709–727. Springer, 2022. 4

2022
[21]

S. Jia, Z. Li, N. Chen, and J. Zhang. Towards visual explainable active learning for zero-shot classification.IEEE Transactions on Visualization and Computer Graphics, 28(1):791–801, 2021. 3

2021
[22]

Joseph, S

K. Joseph, S. Khan, F. S. Khan, and V . N. Balasubramanian. Towards open world object detection. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 5830–5840, 2021. 1, 2, 3, 8

2021
[23]

M. U. Khattak, H. Rasheed, M. Maaz, S. Khan, and F. S. Khan. Maple: Multi-modal prompt learning. InProceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, pp. 19113–19122,
[24]

B. Kim, J. Yu, and S. J. Hwang. Eclipse: Efficient continual learning in panoptic segmentation with visual prompt tuning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3346–3356, 2024. 3

2024
[25]

Lester, R

B. Lester, R. Al-Rfou, and N. Constant. The power of scale for parameter- efficient prompt tuning.arXiv preprint arXiv:2104.08691, 2021. 4

2021 arXiv
[26]

C. Li, C. Wong, S. Zhang, N. Usuyama, H. Liu, J. Yang, T. Naumann, H. Poon, and J. Gao. Llava-med: Training a large language-and-vision assistant for biomedicine in one day.Advances in Neural Information Processing Systems, 36, 2024. 5

2024
[27]

J. Li, R. Selvaraju, A. Gotmare, S. Joty, C. Xiong, and S. C. H. Hoi. Align before fuse: Vision and language representation learning with momentum distillation.Advances in neural information processing systems, 34:9694– 9705, 2021. 2

2021
[28]

X. L. Li and P. Liang. Prefix-tuning: Optimizing continuous prompts for generation.arXiv preprint arXiv:2101.00190, 2021. 4

2021 arXiv
[29]

Liang, B

F. Liang, B. Wu, X. Dai, K. Li, Y . Zhao, H. Zhang, P. Zhang, P. Vajda, and D. Marculescu. Open-vocabulary semantic segmentation with mask- adapted clip. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7061–7070, 2023. 4

2023
[30]

T.-Y . Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick. Microsoft coco: Common objects in context. InCom- puter Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pp. 740–755. ...

2014
[31]

S. Liu, C. Chen, Y . Lu, F. Ouyang, and B. Wang. An interactive method to improve crowdsourced annotations.IEEE transactions on visualization and computer graphics, 25(1):235–245, 2018. 3

2018
[32]

S. Liu, K. Zheng, and W. Chen. Paying more attention to image: A training-free method for alleviating hallucination in lvlms. InEuropean Conference on Computer Vision, pp. 125–140. Springer, 2024. 3

2024
[33]

Y . Liu, Y . Cong, D. Goswami, X. Liu, and J. van de Weijer. Augmented box replay: Overcoming foreground shift for incremental object detection. InProceedings of the IEEE/CVF International Conference on Computer Vision, pp. 11367–11377, 2023. 3

2023
[34]

J. Lu, B. Pan, J. Chen, Y . Feng, J. Hu, Y . Peng, and W. Chen. Agentlens: Visual analysis for agent behaviors in llm-based autonomous systems. IEEE Transactions on Visualization and Computer Graphics, 2024. 3

2024
[35]

Y . Ma, Y . Cao, Y . Hong, and A. Sun. Large language model is not a good few-shot information extractor, but a good reranker for hard samples! arXiv preprint arXiv:2303.08559, 2023. 5

2023 arXiv
[36]

Hello gpt-4o

OpenAI. Hello gpt-4o. https://openai.com/index/ hello-gpt-4o/, 2024. 2024-7-22. 5, 7

2024
[37]

Radford, J

A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al. Learning transferable visual models from natural language supervision. InInternational conference on machine learning, pp. 8748–8763. PMLR, 2021. 2, 7

2021
[38]

Y . Rao, W. Zhao, G. Chen, Y . Tang, Z. Zhu, G. Huang, J. Zhou, and J. Lu. Denseclip: Language-guided dense prediction with context-aware prompting. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 18082–18091, 2022. 2

2022
[39]

Russakovsky, J

O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, et al. Imagenet large scale visual recognition challenge.International journal of computer vision, 115:211– 252, 2015. 3

2015
[40]

Schick and H

T. Schick and H. Schütze. Exploiting cloze questions for few shot text classification and natural language inference.arXiv preprint arXiv:2001.07676, 2020. 4

2001 arXiv
[41]

L. Shu, H. Xu, and B. Liu. Unseen class discovery in open-world classifi- cation.arXiv preprint arXiv:1801.05609, 2018. 2

2018 arXiv
[42]

Thinsungnoena, N

T. Thinsungnoena, N. Kaoungkub, P. Durongdumronchaib, K. Kerdpra- sopb, N. Kerdprasopb, et al. The clustering validity with silhouette and sum of squared errors.learning, 3(7):44–51, 2015. 6

2015
[43]

Uehara and T

K. Uehara and T. Harada. Learning by asking questions for knowledge- based novel object recognition.International Journal of Computer Vision, 132(6):2290–2309, 2024. 3

2024
[44]

Van der Maaten and G

L. Van der Maaten and G. Hinton. Visualizing data using t-sne.Journal of machine learning research, 9(11), 2008. 6

2008
[45]

S. Wang, X. Sun, X. Li, R. Ouyang, F. Wu, T. Zhang, J. Li, and G. Wang. Gpt-ner: Named entity recognition via large language models.arXiv preprint arXiv:2304.10428, 2023. 5

2023 arXiv
[46]

Y . Wang, Z. Yue, X.-S. Hua, and H. Zhang. Random boxes are open- world object detectors. InProceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6233–6243, 2023. 2, 3, 8

2023
[47]

Z. Wang, Y . Li, X. Chen, S.-N. Lim, A. Torralba, H. Zhao, and S. Wang. Detecting everything in the open world: Towards universal object detection. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11433–11443, 2023. 4

2023
[48]

Z. Wang, Y . Lu, Q. Li, X. Tao, Y . Guo, M. Gong, and T. Liu. Cris: Clip- driven referring image segmentation. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 11686–11695,
[49]

Z. Wang, Z. Zhang, C.-Y . Lee, H. Zhang, R. Sun, X. Ren, G. Su, V . Perot, J. Dy, and T. Pfister. Learning to prompt for continual learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 139–149, 2022. 3

2022
[50]

S. T. Wasim, M. Naseer, S. Khan, F. S. Khan, and M. Shah. Vita-clip: Video and text adaptive clip via multimodal prompting. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 23034–23044, 2023. 4

2023
[51]

Y . Wu, Y . Chen, L. Wang, Y . Ye, Z. Liu, Y . Guo, and Y . Fu. Large scale incremental learning. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 374–382, 2019. 2

2019
[52]

J. Xia, Y . Zhang, J. Song, Y . Chen, Y . Wang, and S. Liu. Revisiting dimensionality reduction techniques for visual cluster analysis: An empir- ical study.IEEE Transactions on Visualization and Computer Graphics, 28(1):529–539, 2021. 6

2021
[53]

Xiang, X

S. Xiang, X. Ye, J. Xia, J. Wu, Y . Chen, and S. Liu. Interactive correction of mislabeled training data. In2019 IEEE Conference on Visual Analytics Science and Technology (VAST), pp. 57–68. IEEE, 2019. 3

2019
[54]

Z. Xu, S. Jain, and M. Kankanhalli. Hallucination is inevitable: An innate limitation of large language models.arXiv preprint arXiv:2401.11817,

arXiv
[55]

A. Yang, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Li, D. Liu, F. Huang, H. Wei, et al. Qwen2. 5 technical report.arXiv preprint arXiv:2412.15115, 2024. 5

2024 arXiv
[56]

S. Yang, P. Sun, Y . Jiang, X. Xia, R. Zhang, Z. Yuan, C. Wang, P. Luo, and M. Xu. Objects in semantic topology.arXiv preprint arXiv:2110.02687,

arXiv
[57]

L. Yao, J. Han, Y . Wen, X. Liang, D. Xu, W. Zhang, Z. Li, C. Xu, and H. Xu. Detclip: Dictionary-enriched visual-concept paralleled pre-training for open-world detection.Advances in Neural Information Processing Systems, 35:9125–9138, 2022. 2, 3

2022
[58]

X. Zhao, Y . Ma, D. Wang, Y . Shen, Y . Qiao, and X. Liu. Revisiting open world object detection.IEEE Transactions on Circuits and Systems for Video Technology, 2023. 2

2023
[59]

Zheng, W

J. Zheng, W. Li, J. Hong, L. Petersson, and N. Barnes. Towards open- set object detection and discovery. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3961–3970,
[60]

K. Zhou, J. Yang, C. C. Loy, and Z. Liu. Conditional prompt learning for vision-language models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 16816–16825, 2022. 4

2022
[61]

K. Zhou, J. Yang, C. C. Loy, and Z. Liu. Learning to prompt for vision- language models.International Journal of Computer Vision, 130(9):2337– 2348, 2022. 4

2022
[62]

X. Zhu, W. Su, L. Lu, B. Li, X. Wang, and J. Dai. Deformable detr: Deformable transformers for end-to-end object detection.arXiv preprint arXiv:2010.04159, 2020. 8

2010 arXiv
[63]

Zhuang, R

H. Zhuang, R. He, K. Tong, Z. Zeng, C. Chen, and Z. Lin. Ds-al: A dual-stream analytic learning for exemplar-free class-incremental learning. InProceedings of the AAAI Conference on Artificial Intelligence, vol. 38, pp. 17237–17244, 2024. 2

2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

Z. Bai, P. Wang, T. Xiao, T. He, Z. Han, Z. Zhang, and M. Z. Shou. Hallucination of multimodal large language models: A survey.arXiv preprint arXiv:2404.18930, 2024. 6

arXiv 2024

[2] [2]

Bendale and T

A. Bendale and T. Boult. Towards open world recognition. InProceedings of the IEEE conference on computer vision and pattern recognition, pp. 1893–1902, 2015. 2

work page 1902

[3] [3]

Birhane, A

A. Birhane, A. Kasirzadeh, D. Leslie, and S. Wachter. Science in the age of large language models.Nature Reviews Physics, 5(5):277–280, 2023. 5

work page 2023

[4] [4]

Caron, P

M. Caron, P. Bojanowski, A. Joulin, and M. Douze. Deep clustering for unsupervised learning of visual features. InProceedings of the European conference on computer vision (ECCV), pp. 132–149, 2018. 6

work page 2018

[5] [5]

C. Chen, J. Wu, X. Wang, S. Xiang, S.-H. Zhang, Q. Tang, and S. Liu. Towards better caption supervision for object detection.IEEE Transactions on Visualization and Computer Graphics, 28(4):1941–1954, 2021. 3

work page 1941

[6] [6]

F. D. Davis. Perceived usefulness, perceived ease of use, and user ac- ceptance of information technology.MIS quarterly, pp. 319–340, 1989. 9

work page 1989

[7] [7]

De Angelis, F

L. De Angelis, F. Baglivo, G. Arzilli, G. P. Privitera, P. Ferragina, A. E. Tozzi, and C. Rizzo. Chatgpt and the rise of large language models: the new ai-driven infodemic threat in public health.Frontiers in public health, 11:1166120, 2023. 5

work page 2023

[8] [8]

De Vries, F

H. De Vries, F. Strub, S. Chandar, O. Pietquin, H. Larochelle, and A. Courville. Guesswhat?! visual object discovery through multi-modal dialogue. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5503–5512, 2017. 2

work page 2017