{"id":"728ade84-271f-4483-9755-619b93d3ef4b","arxiv_id":"2507.19870","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"OW-CLIP reports that combining LLM-generated descriptions, user image sorting, and crop-based label smoothing lets a CLIP detector match most state-of-the-art open-world detection accuracy with under 4 percent of the usual training annotations.","lead":"OW-CLIP is a human-in-the-loop system that helps object detectors learn new classes from a few hundred curated images and a handful of LLM-generated text phrases instead of tens of thousands of labels. The authors report about 89 percent of state-of-the-art accuracy at under four percent of the data, and better accuracy than a full-data method given the same small data budget.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 89%/3.8% data-efficiency claim rests on an undisclosed training split: Section 6.1 never states whether the curated images came from the evaluation split, so test-set leakage could inflate Table 1.","rationale":"The reader identified the same load-bearing assumption: Section 6.1's undisclosed split. I agree that this is the weakest point in the argument because every headline number depends on it. If the curated images are drawn from the evaluation split, the central data-efficiency claim collapses; if they are from the training split, the claim is plausible but still needs the artifact to verify. Other concerns (missing unknown-class metrics, single-run results, no code release, abstract overstatement) are secondary and all route through the same need for reproducible evaluation. The degradation ablations and OW-Adapter comparison provide internal support, but they do not resolve the split question because they share the same underlying data provenance. Since the reader's CONDITIONAL verdict already captures this dependency and my check would settle it, no verdict change is needed.","tokens_in":17598,"tokens_out":6320,"duration_ms":66549,"concrete_test":"Obtain the OW-CLIP curated training set for the 20 Task-2 classes (from the released artifact or annotation logs with image filenames/COCO ids) and check every curated image against the exact evaluation split used for Table 1 (VOC 2007 test and COCO val2017, or the split the authors specify). If any curated image overlaps the evaluation split, rerun Table 1 excluding all overlapping images; if the headline 89% figure or the RandBox-S comparison changes materially, the central claim is invalid. If there is no overlap, confirm the curated images came from the standard training split and that RandBox-S was sampled from that same split with matched class balance.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central quantitative claim in the Abstract and Conclusion ('89% of SOTA performance while requiring only 3.8% self-generated data') is supported by Table 1, whose OW-CLIP rows are trained on annotations collected with the visualization system. Section 6.1 says only: 'we processed and annotated images from the conventional OWOD dataset using our visualization system, the annotated results were used as our training data, which differ from the original dataset’s train instances.' It does not state which split (train/val/test) those images came from. In the standard OWOD benchmark, baselines such as RandBox are trained on VOC/COCO training splits and evaluated on held-out test/val splits. If the curated annotations were produced from the same images used for evaluation, the 3.8%-data comparison is not a data-efficiency claim; it becomes test-set contamination, and the 'outperforms SOTA with equivalent data' result could be an artifact of seeing the test distribution during curation. The Delete/Reserve filtering in Section 5.2.2 also makes the curated set non-random, but split overlap is the decisive issue. Because no curated data or code are released and the split is undisclosed, the main quantitative result cannot currently be independently verified.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents OW-CLIP, a visual analytics system for open-world object detection that combines LLM-generated visual feature phrases, interactive image curation into Simple and Hard categories, and multimodal prompt tuning with a novel Crop-Smoothing technique. The central claim is that this human-AI collaboration pipeline lets a frozen-CLIP detector reach 89% of state-of-the-art mAP while using only 3.8% of the training annotations, and that it outperforms RandBox when trained with equivalent data volumes. The evaluation includes a benchmark comparison (Table 1), a comparison with OW-Adapter (Table 2), a degradation study (Table 3), a case study, and a user study.","tokens_in":17867,"tokens_out":3646,"duration_ms":35337,"significance":"If the data-efficiency result is valid, OW-CLIP would be a meaningful step toward practical open-world detection: it offers a plug-and-play prompt-tuning alternative to architectural modifications, and the degradation study in Table 3 gives component-level evidence that Crop-Smoothing, phrase selection, Simple/Hard differentiation, and LLM phrase generation each contribute to performance. The paper also makes concrete, falsifiable claims about annotation effort and usability through the case study and user study. However, the headline 89%/3.8% result rests on data-provenance details that are not currently documented, so the significance cannot be fully assessed without clarification.","major_comments":[{"comment":"The paper states that images from the conventional OWOD dataset were processed and annotated with the visualization system and that the annotated results 'differ from the original dataset’s train instances,' but it never states which split (train/val/test) these images were drawn from. In the standard OWOD protocol, comparison methods are trained on official train splits and evaluated on held-out images; if the curated set includes evaluation images, the 89%/3.8% claim in the Abstract and Conclusion would be inflated by test-set leakage. Please state the exact split, verify disjointness from the evaluation set, and release the curated data or a detailed split description so the comparison can be audited.","section":"Section 6.1, Dataset paragraph"},{"comment":"The reported 3.8% data-efficiency figure depends on the annotation quantity and on the filtering thresholds (ls, hs, lh, hh in Section 5.2.2) being tuned to maximize validation mAP, as shown by the statement that 'optimal performance was achieved with around 220 images and four feature phrases per class.' Because all experiments are single-run, the reader cannot tell whether the 89% number is robust to run-to-run variation or to modest changes in these thresholds. Please provide multiple seeds with mean and standard deviation, and a sensitivity analysis in which annotation count and thresholds are varied.","section":"Table 1 and Section 6.1, Result (4)"},{"comment":"The comparison claims that OW-CLIP 'outperforms SOTA approach when trained with equivalent data volumes,' but RandBox-S is described only as a randomly sampled subset of the original RandBox dataset, with no specification of how the subset was constructed (e.g., which images were selected, how the balanced class distribution was enforced, or how many runs were averaged). Since RandBox-S is trained on random images while OW-CLIP uses curated and filtered images, the row conflates data volume with annotation effort; more detail on the sampling protocol is needed for the equivalence claim to be verifiable.","section":"Table 1, RandBox-S row"}],"minor_comments":[{"comment":"The manuscript contains several typos and grammatical errors, including 'develope' and 'quaily' in the Abstract, 'traning' in Section 6.1, 'multimodel' in Implementation Details, 'visulization' in Section 5.1, and 'Threhold' in Figure 1.","section":"Throughout"},{"comment":"The example text in Figure 11 appears truncated or misformatted ('Zebra? lBlack and white striped pattern'), and the figure caption does not explain what the model predicted instead of the correct class. Please clean the figure and add a short explanation of the misclassification.","section":"Figure 11"},{"comment":"The caption does not define what the Faster-RCNN row represents: is it the standard detector trained on the same known classes, and was it retrained after each task? Please clarify the training protocol for this baseline.","section":"Table 1"},{"comment":"Please report the optimizer settings, prompt-learning rates, and any weight decay or scheduling used for the multimodal prompt tuning, since these details are important for reproducibility.","section":"Section 6.1, Implementation Details"},{"comment":"The definitions of Simple and Hard images are qualitative; please provide the exact threshold ranges used in the experiments so that other researchers can replicate the curation process.","section":"Section 5.2.2"}],"recommendation":"major_revision","confidential_remarks":"The paper fits the scope of the journal, and the system design plus degradation study are substantive. The key blocker is data provenance: the paper never states which image split the curated annotations came from, and the main quantitative claim cannot be verified without this information. I would recommend asking the authors to disclose the split, confirm no overlap with the evaluation set, and release the curated annotation lists or a detailed split description. If the curated data came from the evaluation split, the paper cannot be published as is; if it came from the training split, the claim is likely salvageable with better documentation and multi-run evaluation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague, here's my take on OW-CLIP. The headline claim—89% of SOTA at 3.8% of the data—is the thing to scrutinize. The paper never says which split of the OWOD dataset the curated annotations were pulled from. If they came from the same images used for evaluation, the comparison in Table 1 is not a data-efficiency result; it's test-set contamination. The sentence in Section 6.1 saying the annotations 'differ from the original dataset's train instances' actually raises the alarm rather than quelling it, because it rules out the train split without saying what the alternative was. This is the load-bearing issue, and it makes the central quantitative claim unverifiable as written.\n\nThat's the soft spot in proportion. The rest of the paper is better than average for a system paper. The integration is genuinely new: human-in-the-loop annotation with LLM-generated visual phrases, CLIP prompt tuning, and the Crop-Smoothing trick. Crop-Smoothing is a reparameterization of label smoothing on random crops, but applying it to open-world incremental training is a legitimate idea, and the degradation study (Table 3) gives real internal support—removing it causes a large drop. The comparison to OW-Adapter is the right kind of direct comparison. The limitations section honestly names episode-level class confusion and phrase misguidance, which is more than many papers do.\n\nThe other concerns are real but secondary: no code or curated data released, single-run numbers, annotation counts and thresholds tuned during the experiment, and the 'outperforms SOTA with equivalent data' line overstates because RandBox-S is a random subsample, not the SOTA method itself. If the split issue were resolved, I'd call this a solid system paper with a credible data-efficiency story. As it stands, the central number is unproven.\n\nWho is this for? People working on open-world detection, human-in-the-loop data curation, and visualization-assisted annotation. It deserves a serious referee—the idea is important enough and the system is substantial enough that an editor should send it out, with a strong request to state the split, release the artifacts, and add error bars. If the split is clean, it's a conditional accept; if not, it's a reject for leakage. I'd bring this to reading group for the discussion, but I wouldn't cite the quantitative claim in its current form.","headline":"The 89%/3.8% data-efficiency claim is unverifiable until the authors disclose which split the curated annotations came from, but the system and degradation study are worth taking seriously and the paper merits peer review.","tokens_in":18424,"tokens_out":3263,"would_cite":false,"duration_ms":31632,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A CLIP-based detector reaches 89 percent of state-of-the-art open-world detection performance using only 3.8 percent of the training data.","keywords":["open-world object detection","CLIP","prompt tuning","data-efficient supervision","human-AI collaboration","Crop-Smoothing","visual analytics","annotation quality"],"falsifier":"Reproduce task 2 with curated images drawn strictly from the official training split and evaluated on the official test split; if the 89-percent-of-SOTA figure drops materially under this strict split, the claim depends on evaluation leakage.","tokens_in":17399,"feed_emoji":"🎯","tokens_out":9937,"duration_ms":88806,"temperature":0.7,"pith_summary":"The paper tries to show that the obstacle to practical open-world object detection is not model architecture but the quality and cost of supervision. It proposes a human-AI collaboration system in which a large language model proposes visual feature phrases for a newly discovered class, a CLIP similarity score filters candidate images, and a user marks images as simple or hard. Training then uses prompt tuning with a Crop-Smoothing objective that scales confidence with object completeness. The central quantitative claim is that this recipe reaches about 89 percent of state-of-the-art performance on the second task while using only 3.8 percent of the annotations, and beats the state of the art when both are given the same data volume. If true, open-world detectors can be extended to new classes in minutes of annotation rather than large crowdsourced labeling campaigns.","feed_headline":"CLIP detector reaches 89% of top accuracy with 3.8% of annotations","feed_subtitle":"LLM-generated feature phrases and simple/hard image curation shrink annotation to minutes per class.","key_machinery":"The mechanism that carries the argument is a human-in-the-loop data-curation loop wrapped around frozen CLIP encoders. For a newly named class, an LLM generates candidate visual feature phrases, which a user filters down to a few; the selected phrases are encoded and averaged to initialize a learnable class text vector. Candidate images are scored by CLIP image-text similarity and split into Simple (complete object) and Hard (partial or cluttered) sets, with an interactive density plot guiding threshold choices. During training, Crop-Smoothing randomly crops each image and assigns the ground-truth label a confidence that decreases with the cropping ratio, spreading the leftover probability over other classes, which stops the model from treating local patches as sufficient evidence. Prompt tuning then appends learnable tokens to each ViT layer of the image encoder and per-class context vectors, leaving all pretrained parameters frozen, so each new class is added as a separate plug-in module.","core_discovery":"On the paper's own terms, the central discovery is that a small amount of highly curated dual-modal supervision can stand in for most of the crowdsourced labels that current open-world object detection methods require. In the standard Pascal VOC/MS-COCO open-world setup, the system reports 53.2 mAP in task 1 and 40.0 mAP in task 2, reaching 89 percent of the state-of-the-art result in task 2 while using only 3.8 percent of the annotations, and it outperforms the state of the art when both use the same data volume. The authors attribute the result to three interacting components: class-specific feature phrases that initialize text prompts, a Crop-Smoothing objective that ties classification confidence to object completeness, and human filtering that separates images into Simple and Hard sets. Ablation results are presented as evidence that each component contributes, with Crop-Smoothing and image differentiation producing the largest performance drops when removed.","pith_inferences":["A natural extension beyond the paper is to automate the Simple/Hard split with an object-completeness estimator, since the ablation attributes most of the gain to the human image-differentiation step; if an automatic proxy works, the pipeline becomes nearly annotation-free.","Because image filtering is driven by CLIP similarity, the system inherits CLIP's embedding biases; classes that are visually distant from CLIP's pretraining vocabulary may yield poor candidate sets, so the 3.8-percent data claim is most likely to generalize for classes close to CLIP's distribution.","The paper's own limitation discussion notes that LLM phrases can be redundant or misleading, such as emphasizing striped texture for zebra; a testable fix is to add a phrase-diversity penalty or rank phrases by mutual information before presenting them to the user.","The equal-data comparison with the state of the art uses randomly sampled images for the baseline; a further stress test would compare against an active-learning baseline that also selects hard examples, isolating whether the gain comes from curation or from the Crop-Smoothing objective."],"forward_implications":["A new class can be added with roughly 220 curated images and four selected feature phrases, taking about 4 minutes for simple classes and 6-9 minutes for more complex ones.","Because the proposal generator and pretrained parameters stay frozen, the same detector can be extended to new classes without retraining old ones or changing the architecture.","The forgetting rate for previously learned classes stays at 8 percent after adding 20 classes, compared with 12 percent for the SOTA method, so incremental updates are less destructive.","Removing Crop-Smoothing costs about 9.8 mAP points in task 1 and 6.7 points in task 2, making the completeness-aware confidence the largest single contributor among the system's components.","Classes learned in different episodes are not trained to be distinguished from each other, so users must group confusable classes in the same episode to avoid confusion."],"supporting_citations":[{"why":"Supplies the frozen CLIP image and text encoders that the prompt-tuning and similarity filtering build on.","marker":"[37]"},{"why":"Supplies the frozen Faster R-CNN proposal generator that produces object proposals and unknown candidates.","marker":"[11]"},{"why":"Defines the state-of-the-art baseline whose performance and data volume the paper compares against; the equal-data variant is the direct control.","marker":"[46]"},{"why":"Provides the OW-Adapter comparison system and its annotation setup used in the case study and comparative table.","marker":"[19]"},{"why":"Provides the visual prompt tuning method that the paper adapts for deep prompt tuning of the image encoder.","marker":"[20]"},{"why":"Supplies the LLM used to generate the candidate visual feature phrases in the text-modality annotation workflow.","marker":"[36]"},{"why":"Defines the open-world object detection task and the baseline experimental setting that the paper follows.","marker":"[22]"}],"fun_headline_variants":["Open-world detection: 3.8% labels, 89% of SOTA","Human-AI curation cuts annotation 26x, keeps 89% of SOTA","Data-efficient OWOD: 3.8% annotations hit 89% of SOTA","3.8% of labels yield 89% of top accuracy in open-world detection"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The comparison assumes that the images users curated, and the Delete/Reserve filtering they applied, did not remove difficult cases from the evaluation split, since the paper does not state which image split its curated annotations came from.","fun_headline_variants_meta":{"raw":{"variants":["Open-world detection: 3.8% labels, 89% of SOTA","Human-AI curation cuts annotation 26x, keeps 89% of SOTA","Data-efficient OWOD: 3.8% annotations hit 89% of SOTA","3.8% of labels yield 89% of top accuracy in open-world detection"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000857,"raw_usage":{"total_tokens":3746,"prompt_tokens":994,"completion_tokens":2752,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":610,"completion_tokens_details":{"reasoning_tokens":2659}},"tokens_in":610,"tokens_out":2752,"duration_ms":21105,"temperature":1.0,"reasoning_tokens":2659,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T17:52:00.364739+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Reproduce task 2 with curated images drawn strictly from the official training split and evaluated on the official test split; if the 89-percent-of-SOTA figure drops materially under this strict split, the claim depends on evaluation leakage.","supporting_citations":[{"cited_title":"Radford, J","cited_arxiv_id":null,"evidence_quote":"Supplies the frozen CLIP image and text encoders that the prompt-tuning and similarity filtering build on."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the frozen Faster R-CNN proposal generator that produces object proposals and unknown candidates."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the state-of-the-art baseline whose performance and data volume the paper compares against; the equal-data variant is the direct control."},{"cited_title":"Jamonnak, J","cited_arxiv_id":null,"evidence_quote":"Provides the OW-Adapter comparison system and its annotation setup used in the case study and comparative table."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the visual prompt tuning method that the paper adapts for deep prompt tuning of the image encoder."},{"cited_title":"Hello gpt-4o","cited_arxiv_id":null,"evidence_quote":"Supplies the LLM used to generate the candidate visual feature phrases in the text-modality annotation workflow."},{"cited_title":"Joseph, S","cited_arxiv_id":null,"evidence_quote":"Defines the open-world object detection task and the baseline experimental setting that the paper follows."}],"review_version":1}