{"id":"96d33736-f6aa-40e4-838d-0ce9f233e689","arxiv_id":"2411.15620","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"FOCUS masks everything outside a user-selected region with SAM, uses GPT-4o to propose an object list, and feeds that list to GroundingDINO or OWLv2 to output bounding boxes.","lead":"FOCUS combines SAM region masking, a GPT-4o object-proposal prompt, and an open-vocabulary detector so users can ask for fine-grained detections inside a chosen box. The quantitative gains are computed against the very GPT-4o proposal list that the pipeline itself generates, which makes the evaluation self-referential rather than independent.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Evaluation uses the VLM's own proposal list as ground truth (Sec. 4.1), so Table 1 measures self-consistency between FOCUS modules, not detection accuracy; the central enhancement claim lacks independent support.","rationale":"The reader's weakest assumption identifies the same circularity. I agree: this is the single most load-bearing concern because every reported quantitative result (Table 1, Figures 3 and 6) uses the VLM proposal list as ground truth, and the VLM is part of the proposed pipeline. Without human-verified ground truth, the numbers cannot distinguish true detection improvements from the pipeline's internal consistency. The ablation in Sec. 5.4 also admits that direct comparisons are 'infeasible due to variations in VLM proposals,' further confirming the metric's fragility. The paper's contribution could be legitimate, but the evidence as presented does not support the central claim. No change to the reader's REJECT verdict is needed.","tokens_in":9001,"tokens_out":3593,"duration_ms":31196,"concrete_test":"Select a random subset of the test images (e.g., 100 COCO human images), have annotators mark all granular objects with bounding boxes (or use PASCAL-Part labels). Run FOCUS (SAM mask + GPT-4o proposals + GroundingDINO) and the baseline GroundingDINO with the same user prompt, and compute F1 with IoU>=0.5 against the human boxes instead of the VLM list. If the human-grounded F1 improvement is not reproduced (or the gap shrinks to within noise), the circular evaluation in Table 1 is the source of the reported gains.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The quantitative basis for the central claim is Equation (1), computed between Lp (object list proposed by GPT-4o) and Ld (detector outputs). Section 4.1 explicitly states that the VLM proposal list is assumed to be ground truth. Because GPT-4o is a component of FOCUS, the reported F1 scores reflect how well the detector agrees with an internal module, not how accurately objects are detected relative to human labels. The baseline detectors also receive the same Lp as input text queries, so any gain is specific to the masked image making it easier for the detector to match the VLM's own list. Human-level correctness of the proposed objects (omissions, hallucinations, naming) is never checked. The claim that FOCUS 'effectively enhances detection capabilities' therefore rests entirely on a self-consistency measure, and the large margins in Table 1 cannot be interpreted as detection accuracy improvements.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FOCUS, a zero-shot pipeline that combines SAM, a vision-language model (GPT-4o), and an open-vocabulary detector. The user supplies a bounding box and a natural-language prompt; SAM masks everything outside the box, the VLM proposes an object list, and the detector localizes those objects. The authors report F1 scores on COCO person images and PASCAL VOC vehicle images, claiming that FOCUS improves granular detection over GroundingDINO, OWLv2, and OpenSeeD, and that it is robust to confidence thresholds and image complexity.","tokens_in":9110,"tokens_out":4831,"duration_ms":46256,"significance":"If the reported gains were supported by independent ground truth, FOCUS would be a practically useful, training-free framework for prompt-controlled fine-grained detection, and the modular design (model-agnostic across all three stages) is a genuine strength. The qualitative figures illustrate appealing flexibility, such as switching between instance-level and body-part-level detection by editing a text prompt. However, the central quantitative claim is currently not established because the evaluation is circular: the same VLM that proposes objects inside FOCUS is used to define ground truth in the F1 computation. The paper also provides no localization metric, no error bars, no image counts, and an incomplete ablation (an undefined 'N%'). For these reasons, the significance of the contribution cannot be assessed from the present manuscript.","major_comments":[{"comment":"The evaluation is circular and does not measure detection accuracy. Eq. (1) computes F1 between Lp (the VLM proposal list) and Ld (the detector outputs), and Section 4.1 states that 'the object list proposal made by the VLM' is assumed to be ground truth. Since the same VLM, GPT-4o, is a component of FOCUS (Section 3.2), all F1 scores in Table 1 measure self-consistency between an internal proposal module and the detector, not correctness relative to human labels. For example, the 0.835 vs. 0.465 difference for GroundingDINO at threshold 0.2 may only reflect that the masked image makes it easier for the detector to reproduce the VLM's own list. The authors should evaluate against human-annotated object lists, or at least against a VLM not used inside FOCUS, and report agreement with those independent references.","section":"Section 4.1, Eq. (1)"},{"comment":"No box-level localization metric is reported. The paper claims that FOCUS 'locates' granular components and that the Entity Identification module outputs final bounding boxes, but Eq. (1) only compares label lists; it never checks whether a detected box actually covers the corresponding object. Without an IoU-based metric (e.g., mAP at IoU=0.5, or at least box-center accuracy), the reported F1 values cannot support the claim that detection capabilities are enhanced, because a detector could produce the right labels with wrong boxes and still score perfectly under Eq. (1).","section":"Section 3.3 and Section 4.1"},{"comment":"The ablation study is incomplete: it states that 'the FOCUS framework improves performance on the granular detection task by N% compared to the baseline itself,' where N is never defined. This is not a quantitative result and cannot support the claim that SAM-based masking is responsible for the observed gains. The authors must provide the actual value, the uncertainty estimate, and the evaluation protocol used for the ablation.","section":"Section 5.4"},{"comment":"Essential experimental details are missing, making the results non-reproducible. The manuscript does not report the number of test images per class, the exact prompt templates used for Ptask and Padd, the GPT-4o version or inference settings, or the procedure behind the 'top 10 items with highest score discrepancies' mentioned in Section 5.3. Table 1 and Figure 6 report no error bars or significance tests across threshold values, so the claimed consistency and threshold robustness are not substantiated.","section":"Section 4.2 and Section 5.3"}],"minor_comments":[{"comment":"The abstract contains a typo: 'Object Re Cognition' should be 'Object Recognition'.","section":"Abstract"},{"comment":"The term 'spacial relations' should be 'spatial relations'.","section":"Section 3.3"},{"comment":"In the caption, 'Top later shows instance detection; bottom later shows anatomic detection' should read 'Top layer' and 'bottom layer'.","section":"Figure 5 caption"},{"comment":"References [15] and [16] are the same CLIP paper by Radford et al.; one should be removed or merged to avoid a duplicate citation.","section":"References"},{"comment":"The phrase 'humane and inhumane target categories' is awkward; consider replacing with 'human-related and non-human target categories'.","section":"Section 5.1"}],"recommendation":"major_revision","confidential_remarks":"The circular evaluation is severe: the central quantitative claim rests entirely on a self-consistency measure between FOCUS's own VLM proposals and its detector. I would require independent human-annotated object lists and box-level localization metrics before considering the paper for acceptance. If the authors cannot provide such an evaluation, the paper should be rejected. The placeholder 'N%' in the ablation and the missing experimental details further indicate that the manuscript is not yet in a publishable state."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe short version: this is a reasonable engineering assembly (SAM mask, GPT-4o proposals, open-vocabulary detector), but the headline numbers in Table 1 do not measure detection accuracy. Equation (1) computes F1 between the detector output and the VLM-generated object list, and Section 4.1 explicitly says that list is assumed to be ground truth. Since the same GPT-4o model generates that list inside FOCUS, the 0.835 vs 0.465 F1 is at best a self-consistency gain, not evidence that FOCUS detects granular parts better against independent human judgment.\n\nWhat is genuinely useful: the idea of using a user-drawn box to isolate a region before asking a VLM to enumerate sub-objects is a nice way to get flexible granularity, and the qualitative figures do show the pipeline picking out belts, sunglasses, and similar items. The design is model-agnostic, which is a plus. The paper is also honest about the evaluation assumption, which is more than some papers manage, but that honesty exposes the gap.\n\nSoft spots, in order of severity. First, the circularity is load-bearing: every F1 number in Table 1 is conditional on the VLM's list being correct. If GPT-4o omits or hallucinates objects, the score changes without any change in real detection quality. Second, there are no error bars, no image counts, and no box-level IoU evaluation. Third, the ablation in Section 5.4 reports \"N%\" instead of a number. Fourth, the novelty claim of being \"the first to integrate VLMs in object detection\" ignores Grounded SAM and similar assemblies; the related work even cites [20] but does not discuss the overlap. Fifth, no code or data, so nothing is independently checkable.\n\nThe paper is not nonsense; it describes a plausible pipeline with a clear use case. But the submitted evidence does not justify the central claim, and the fix—human-annotated ground truth, error bars, a real ablation—is substantial enough that I would rather wait for a revised version than send this one to reviewers. If it did go to review, the referee would need to demand a full re-evaluation, which is effectively a rewrite.\n\nRecommendation: desk reject, with an encouraging note to resubmit after a human-labeled evaluation and proper ablations. Not worth citing in its current form, though I'd keep an eye on the authors' future work.","headline":"A sensible three-stage recipe, but Table 1 measures self-consistency with the VLM's own proposals rather than detection accuracy, so the central claim is unsupported as submitted.","tokens_in":9699,"tokens_out":2096,"would_cite":false,"duration_ms":19620,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"FOCUS wraps open-vocabulary detectors with SAM and a VLM, lifting fine-grained detection F1 from 0.465 to 0.835.","keywords":["open-vocabulary object detection","fine-grained detection","vision-language models","prompt-guided detection","region isolation","segment anything","grounding dino","zero-shot detection"],"falsifier":"Take a set of images, have a human annotator list the fine-grained objects they consider present inside the user box, and compute F1 for GroundingDINO and GroundingDINO+FOCUS against that human list. If the FOCUS gain over baseline shrinks to near zero, or if human precision on FOCUS outputs is low because the VLM proposed objects that are absent, then the claimed enhancement is an artifact of self-consistent evaluation rather than a real detection improvement.","tokens_in":8728,"feed_emoji":"🎯","tokens_out":6171,"duration_ms":48367,"temperature":0.7,"pith_summary":"FOCUS is a three-stage pipeline that wraps an open-vocabulary object detector with two foundation models: a segmenter that masks everything outside a user-selected region, and a vision-language model that generates a natural-language object list from the masked image. The paper claims that this combination lets detectors find small constituent parts of larger objects far more reliably, raising GroundingDINO's F1 from 0.465 to 0.835 on granular-person detection at threshold 0.2. It also lets users change granularity by editing a text prompt, for example switching from detecting worn items to detecting body parts on the same image. If the claim holds, FOCUS offers a model-agnostic way to add user-controllable fine-grained recognition to existing detectors without retraining.","feed_headline":"SAM + VLM boosts fine-grained detection F1 from 0.465 to 0.835.","feed_subtitle":"A three-stage pipeline lets users steer detection with natural-language prompts while finding small object parts.","key_machinery":"The load-bearing mechanism is the three-stage FOCUS pipeline: (1) Region Isolation, where SAM takes the user's bounding box $B_{\\text{input}}$ and returns a binary mask $M_{\\text{binary}}$, producing the masked image $I_{\\text{atten}}$ that blanks out everything outside the target; (2) Proposal Extraction, where a VLM prompted with a free-form task prompt $P_{\\text{task}}$ and formatting instructions $P_{\\text{add}}$ returns an object list $L_p$; and (3) Entity Identification, where an open-vocabulary detector consumes $I_{\\text{atten}}$ and $L_p$ to output bounding boxes $B_{\\text{output}}$. The performance is scored by $F_1(L_p, L_d)$, the harmonic mean of recall and precision between the VLM proposal and the detected list. The key mechanism is that masking removes the large-object bias that causes detectors to overlook small nested components.","core_discovery":"The paper's central claim is that the failure of open-vocabulary detectors on fine-grained parts is not mainly a detector-capacity problem; it is a focusing and proposal problem. By first applying a binary mask from SAM to the user-designated bounding box, the pipeline removes distracting context so the detector attends to the target. A VLM then proposes the candidate object list $L_p$ from this masked image, and an open-vocabulary detector locates those candidates, producing $L_d$. The paper reports that this yields consistent gains across detectors and object types, with the largest improvement on granular human targets: at a 0.2 threshold, GroundingDINO+FOCUS reaches 0.835 F1 versus 0.465 for GroundingDINO. The method is intended to be zero-shot and model-agnostic, with any VLM or open-vocabulary detector substitutable in the pipeline.","pith_inferences":["Because the evaluation treats the VLM's proposal as ground truth, the reported F1 measures how well the detector reproduces the VLM's object list; it would be informative to re-run with human-annotated fine-grained labels to see how much of the gain reflects agreement with human perception.","Masking everything outside the user box could suppress context that helps identify objects that extend beyond the box or that are semantically defined by surroundings; a targeted study on truncation cases would bound this effect.","Prompt sensitivity is an implicit free parameter: the same pipeline could produce different object lists for differently worded prompts, so systematic prompt variation is a natural testable extension.","If VLM hallucination is common in cluttered scenes, errors propagate into the detector; measuring precision against human labels would reveal whether FOCUS trades recall for hallucination-driven false positives."],"forward_implications":["Users can re-target the same image to different granularities by editing a natural-language prompt, e.g., from physical objects worn by a person to visible body parts.","The pipeline is model-agnostic: the segmentation module, VLM, and detector can each be swapped, so improvements should transfer to newer foundation models as they appear.","FOCUS-equipped detectors are more resistant to confidence-threshold changes, meaning detections remain useful even when the cutoff is low.","The gains hold across both human and non-human targets, with vehicle-detection improvements becoming clearer at higher score thresholds.","Because the same VLM proposal list is fed to baselines, the comparison isolates the effect of the region-isolation mask."],"supporting_citations":[{"why":"Supplies SAM, the segmenter that masks the region of interest in the Region Isolation stage.","marker":"[7]"},{"why":"Supplies SAM 2 as an alternative segmentation backbone for the same masking role.","marker":"[17]"},{"why":"GroundingDINO is the main open-vocabulary detector and the strongest baseline whose F1 rises from 0.465 to 0.835 with FOCUS.","marker":"[11]"},{"why":"GPT-4o is the VLM that generates the object proposals $L_p$ from the masked image.","marker":"[14]"},{"why":"OWLv2 serves as the second detector baseline, showing a smaller but positive FOCUS gain.","marker":"[13]"},{"why":"COCO 2017 validation set provides the humane-target images used in the granular-person experiments.","marker":"[10]"},{"why":"PASCAL VOC 2012 validation set provides the inhumane-target (vehicle) images used in the cross-target experiments.","marker":"[2]"}],"fun_headline_variants":["User-guided FOCUS lifts fine-grained detection F1 from 0.465 to 0.835","FOCUS: Natural language prompts guide detection of small object parts","Zero-shot FOCUS pipeline boosts fine-grained part detection via user prompts","From 0.465 to 0.835: FOCUS sharpens open-vocabulary part detection"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the vision-language model's proposed object list can stand in for ground truth, so the F1 score in Equation (1) is assumed to reflect how well objects are actually detected rather than how well the detector agrees with that same VLM.","fun_headline_variants_meta":{"raw":{"variants":["User-guided FOCUS lifts fine-grained detection F1 from 0.465 to 0.835","FOCUS: Natural language prompts guide detection of small object parts","Zero-shot FOCUS pipeline boosts fine-grained part detection via user prompts","From 0.465 to 0.835: FOCUS sharpens open-vocabulary part detection"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000573,"raw_usage":{"total_tokens":2679,"prompt_tokens":886,"completion_tokens":1793,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":502,"completion_tokens_details":{"reasoning_tokens":1700}},"tokens_in":502,"tokens_out":1793,"duration_ms":13241,"temperature":1.0,"reasoning_tokens":1700,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T14:05:44.372834+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a set of images, have a human annotator list the fine-grained objects they consider present inside the user box, and compute F1 for GroundingDINO and GroundingDINO+FOCUS against that human list. If the FOCUS gain over baseline shrinks to near zero, or if human precision on FOCUS outputs is low because the VLM proposed objects that are absent, then the claimed enhancement is an artifact of self-consistent evaluation rather than a real detection improvement.","supporting_citations":[{"cited_title":"Grounding dino: Marrying dino with grounded pre-training for open-set object detection","cited_arxiv_id":null,"evidence_quote":"GroundingDINO is the main open-vocabulary detector and the strongest baseline whose F1 rises from 0.465 to 0.835 with FOCUS."},{"cited_title":"Scaling open-vocabulary object detection","cited_arxiv_id":null,"evidence_quote":"OWLv2 serves as the second detector baseline, showing a smaller but positive FOCUS gain."}],"review_version":1}