REVIEW 2 major objections 6 minor 23 references
Detector Confidence Signals Presence Rather Than Occlusion in Cluttered Manipulation
T0 review · 2 major / 6 minor · reviewed 2026-08-02 · deepseek-v4-flash
Pith's one-line read Open-vocabulary detector confidence signals category presence, not target visibility.
desk verdict A well-designed negative result showing detector confidence tracks category presence, not target visibility, with real consequences for active-perception gating — the instance-level caveat is real but already inside the paper's stated scope. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central machinery is the geometry-oracle audit: a detector-free measurement of true visibility V(p), the pixel count of the target's ground-truth mask in the rendered view at pose p, compared against the detector's confidence c(p) and its top-scoring region r(p) on the same view. The audit pairs a fixed default view with the best of 24 candidate views on a hemisphere, and attributes a detection as on-target when its overlap with the target mask exceeds one half. This separation of geometry from appearance is what lets the paper establish that c(p) stays high while V(p) collapses, and that r(p) rests on same-category distractors. The audit is cheap because a simulator provides the segment
What would settle it
A factorial experiment would settle it: fix the target's visible pixels at a constant value and vary the number of same-category distractors in view. If confidence tracks category presence, mean confidence should rise monotonically with distractor count while visibility is unchanged; if confidence tracks occlusion, it should remain flat. The paper's ring-layout control (confidence drops while visibility stays constant) and randomized frontal layout (confidence rises while visibility falls) already bracket this, but a direct manipulation of distractor count at constant target visibility would i
Extended reading notes
Core claim
The central discovery is that open-vocabulary detector confidence is a presence signal, not an occlusion or visibility signal. In controlled cluttered scenes, the target's true visible pixel count fell with added distractors from covering every scene to one in eight, while a common detector's mean confidence stayed at 0.39–0.40 and its resolved rate stayed above 0.92; by pixel-level attribution, the fraction of confident detections that actually overlapped the target dropped from near one to near zero, because the detector fired on another object of the same category. On real video the detector reported the occluded target present in 99% of frames and localized to a different instance in 94%
Load-bearing premise
The central finding assumes the system's task is to locate a specific target instance while the detector is prompted with only a category name, so firing on a same-category distractor counts as a failure; for a purely category-level question such as 'is any soup can present?', the detector's response would not be an error.
Editorial extensions
If this is right
- Confidence thresholds used to filter detections will admit false positives whenever another object of the named category is in view, because the score does not fall when the target itself is hidden.
- Any evaluation of open-vocabulary detection or active perception that scores by category confidence will understate the benefit of resolving occlusion; in the paper's active-perception setting the confidence-based benefit is 8 points against an 88-point oracle benefit.
- A gate that moves the camera only when confidence is low will stay put in heavily occluded scenes, since confidence stays high; the safe policy is to treat heavy clutter as a reason to look.
- Benchmarks built with a ring of distractors around the target do not actually occlude it, so results on such layouts may credit active perception with resolving occlusion that was never present.
- In a downstream grasping test, a high-confidence fixed view left the target localizable in only 5% of scenes, while moving first made it localizable in every scene; yet even then residual depth bias kept end-to-end grasp success at 0%.
Reading between the lines
- If confidence is presence rather than visibility, then referring-expression grounding and instance retrieval that reuse the same score likely inherit the same identity confusion in cluttered scenes, even outside manipulation.
- A direct testable extension: hold target visibility fixed and vary the number of same-category distractors; the presence hypothesis predicts confidence rises with distractor count, while the visibility hypothesis predicts it does not.
- The instance-level task assumption is load-bearing: if the downstream goal is only to know whether any soup can is present, the detector's distractor responses would be correct rather than erroneous. The paper argues active-perception systems make the category-to-instance inference, but the reported failure is conditional on that premise.
- The residual localization error after moving suggests a limit beyond confidence: mask-plus-depth localization is biased because an occluder sits at the target's center pixel, so active perception alone may not reach grasp tolerance without multi-view fusion or contact-rich search.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper proposes a geometry-oracle audit to test whether open-vocabulary detector confidence tracks the true visibility of a specific target object under occlusion. In rendered manipulation scenes, the oracle counts the target's visible pixels from a ground-truth segmentation, while detectors (Grounding DINO, OWLv2, SAM3) are queried with category labels. The results show that as true target visibility falls from ~100% to ~12% of scenes, detector confidence remains nearly flat (≈0.40) and the resolved rate stays above 90%, with top detections increasingly landing on same-category distractors (up to 88% off-target). This decoupling is shown across nine categories, two simulators, and real video (DAVIS), with some variation in degree (OWLv2 and SAM3 lower their scores under occlusion but still fail to localize the target). The paper quantifies downstream consequences: a confidence-based active-perception benefit metric understates the value of moving the camera by about a factor of ten, and a confidence-based gate fails to signal occlusion. The authors also show that a ring occluder layout does not actually occlude the target, so confidence drops there measure clutter rather than occlusion. They release (claim) a benchmark and recommend target-grounded signals for gating and evaluation.
Significance. The paper provides a well-controlled, detector-free empirical demonstration that open-vocabulary detector confidence is not a reliable proxy for the visibility of a specific target instance in clutter. The geometry-oracle protocol is a strong methodological contribution, and the robustness checks (thresholds, multiple detectors, two simulators, real video) support the core finding. If accepted, the paper would serve as an important caution for the robotics and vision communities: confidence scores should be interpreted as category-presence signals, not instance-visibility signals, in tasks such as active perception, grounding, and retrieval. The paper is also honest about limitations, including the residual localization error and the fact that end-to-end grasping still fails even after active perception. The main weakness is a scope condition: the claim that the detector is 'wrong' depends on an instance-level downstream task, while the queries are category names. For a category-level presence task, firing on another instance of the same category is arguably correct. The paper should more carefully delimit its claims and provide evidence that the cited deployed systems actually mak
major comments (2)
- [§1, §5] The central conclusion—'Detector confidence answers whether something matching the query appears anywhere in the view, not whether the specific target is visible at a specific place'—is conditional on an instance-level downstream task. However, the paper motivates the 'matters' claim by citing VISO-Grasp and ActiveGrasp without verifying that these systems actually prompt with category names and read the score as an instance-visibility signal. A category-level query is expected to fire on a same-category distractor; the paper even shows that the detector is reading a different can. The paper should either (a) provide evidence from the cited systems that they perform the category-to-instance inference, or (b) explicitly scope the conclusion to instance-level tasks and soften the title/abstract to avoid implying that the detector is malfunctioning for category-presence detection. This is l
- [Abstract, §1, §4.4] The paper repeatedly states that it 'releases the controlled benchmark' but provides no artifact, repository link, or data-availability statement. Without the benchmark, the third contribution is unverifiable and the experiments cannot be exactly reproduced from the description alone. Please add a working link or state that the data will be made available upon acceptance; if no artifact is intended, the release claim should be removed.
minor comments (6)
- [§4.1] Typo: 'Thedetectorisalmostblindtothisocclusion' should have spaces. Similar spacing issues appear in §2 ('compareintegrationparadigmsandreport'). A proofread pass is needed.
- [§3.1, §4.1] The term 'resolved' is used to mean that confidence exceeds a threshold, but this could be misread as actual target resolution. Consider using 'detector-resolved' or adding a qualifier to distinguish it from oracle visibility.
- [§4.1] The phrase 'a confidence gate fires exactly when the object is hidden' is misleading: the gate fails by staying put (always confident), not by triggering at hidden times. Rephrase to 'a confidence gate reports the target as visible and fails to trigger a move when the object is hidden.'
- [§4.3, Table 4] SAM3 is described as a 'detector' throughout, but it is a promptable segmentation model with a presence score. Please clarify this in the text to avoid conflating detection and segmentation.
- [Table 3] The large gap between GDINO (AUC 0.50) and OWLv2 (AUC 0.76) for confidence-based gating deserves a brief explanation in the text, e.g., linking to the partial correlation of OWLv2 with visibility reported in Table 4.
- [§4.5, §6] The downstream grasping experiment shows 0% end-to-end grasp success even after moving the camera. The paper honestly acknowledges this, but the statement in §5 that 'the safe policy is to treat heavy clutter as a reason to look' could imply a validated solution. Consider explicitly stating that active perception improves localizability but not grasp success within the tested pipeline.
Circularity Check
No significant circularity: the central claim is an empirical audit against an independent geometry oracle.
full rationale
The paper's central claim is an empirical finding, not a derivation from its own assumptions. V(p) is computed from ground-truth segmentation masks that no detector influences, c(p) comes from public detector checkpoints, and the correlation, resolved rates, and IoU attributions are measurements rather than fitted parameters or equations. The benefit metric B_s in Eq. (1) is a definition used to compare fixed versus active strategies, and no detector parameter or oracle threshold is fitted to the reported outcomes. The only arguable premise is that the task requires locating a specific target instance while queries are category labels; this is a modeling assumption stated in §3.1 and motivated in §1 and §5, not a circular derivation. Self-citations (He et al., 2025; 2026) appear only in Related Work as parallel policy-learning lines and do not carry the central argument. Limitations are explicitly acknowledged (§6: the oracle is approximate on real video, and residual localization error leaves end-to-end grasp at 0%), which reinforces the empirical, not definitional, nature of the audit. No load-bearing step reduces to its own inputs.
Assumptions & free parameters
free parameters (3)
- visibility threshold v0 =
40 pixels
- GDINO confidence threshold tau =
0.35
- on-target overlap threshold =
IoU > 0.5
assumptions (5)
- domain assumption Ground-truth segmentation mask M*(p) is a detector-free oracle for target visibility, with V(p)=|M*(p)|.
- domain assumption The category-name prompt is the intended query for locating a specific target instance.
- domain assumption Occluder counts, layouts, and the 24-candidate hemisphere view set model realistic active-perception decision points.
- domain assumption A detection is a false positive relative to the target when IoU(r, M*) <= 0.5.
- domain assumption A DAVIS frame is occluded when the object's visible area falls below 30% of its per-clip median.
Cite this review
Pith. "Pith review of Detector Confidence Signals Presence Rather Than Occlusion in Cluttered Manipulation." pith.science (2026). https://pith.science/paper/LSTP73QD
@misc{pith2026260713361,
author = {Pith},
title = {Pith review of: Detector Confidence Signals Presence Rather Than Occlusion in Cluttered Manipulation},
year = {2026},
howpublished = {\url{https://pith.science/paper/LSTP73QD}},
note = {Machine review of arXiv:2607.13361}
}
read the original abstract
Occlude a named object until about an eighth of it remains visible, and an open-vocabulary detector's confidence that the object is present barely changes; as the clutter around it grows the confidence can even rise. On real video the detector still reports the object present in 99% of occluded frames, on another instance of the same category. This matters because that confidence is widely read as a visibility signal, used to threshold detections, evaluate open-vocabulary detectors, ground language, retrieve instances, and gate active perception. We audit whether it reflects occlusion by pairing every view with a geometry-segmentation oracle that gives detector-free ground-truth visibility. As true visibility falls from every scene to one in eight, the confidence stays nearly constant and uncorrelated with visibility, and the detector reports the target present in about nine of ten scenes, firing on same-category distractors: it signals that the category is present somewhere, not that the specific target is visible. The failure holds across three detectors (Grounding DINO, OWLv2, and Segment Anything Model 3), nine object categories, two simulators with different renderers and object sets, built and natural occlusion, and real video. Two consequences follow: a confidence-based metric understates the value of resolving occlusion by about ten times (8 against 88 points in our active-perception setting), and a confidence-based gate fires exactly when the object is hidden. No single-view signal we tried, including a realizable localization check, flags the occlusion, because the occluders sit where the target is. We connect the effect to detector miscalibration and object hallucination, release the controlled benchmark, and recommend target-grounded signals for gating and evaluation.
Figures
Reference graph
Works this paper leans on
-
[1]
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages=
Closed-loop next-best-view planning for target-driven grasping , author=. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , pages=
-
[2]
Shi, Yitian and Wen, Di and Chen, Guanqi and Welte, Edgar and Liu, Sheng and Peng, Kunyu and Stiefelhagen, Rainer and Rayyes, Rania , journal=
-
[3]
Towards Autonomous Robotic Systems (TAROS) , pages=
Few-Shot Transfer Learning for Deep Reinforcement Learning on Robotic Manipulation Tasks , author=. Towards Autonomous Robotic Systems (TAROS) , pages=
-
[4]
Sample-efficient Low-level Motion Planning for Robotic Manipulation Tasks via Zero-shot Transfer Learning , author=. arXiv:2606.06041 , year=
-
[5]
Lei, Boshu and Jiang, Wen and Daniilidis, Kostas , booktitle=
-
[6]
Grounding
Liu, Shilong and Zeng, Zhaoyang and Ren, Tianhe and Li, Feng and Zhang, Hao and Yang, Jie and Li, Chunyuan and Yang, Jianwei and Su, Hang and Zhu, Jun and Zhang, Lei , journal=. Grounding
-
[7]
Advances in Neural Information Processing Systems (NeurIPS) , year=
Scaling open-vocabulary object detection , author=. Advances in Neural Information Processing Systems (NeurIPS) , year=
-
[8]
arXiv:2511.16719 , year=
Show all 23 references
-
[9]
arXiv:2603.10340 , year=
Overcoming visual clutter in vision-language-action models via concept-gated visual distillation , author=. arXiv:2603.10340 , year=
-
[10]
arXiv:2505.15685 , year=
From grounding to manipulation: Case studies of foundation model integration in embodied robotic systems , author=. arXiv:2505.15685 , year=
-
[11]
Score the steps, not just the goal:
ElMallah, Ramy and Chhajer, Krish and Lee, Chi-Guhn , booktitle=. Score the steps, not just the goal:
-
[12]
The 2017
Pont-Tuset, Jordi and Perazzi, Federico and Caelles, Sergi and Arbel. The 2017. arXiv:1704.00675 , year=
2017 arXiv
-
[13]
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year=
Grounded language-image pre-training , author=. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year=
-
[14]
European Conference on Computer Vision (ECCV) , year=
Simple open-vocabulary object detection with vision transformers , author=. European Conference on Computer Vision (ECCV) , year=
-
[15]
Cheng, Tianheng and Song, Lin and Ge, Yixiao and Liu, Wenyu and Wang, Xinggang and Shan, Ying , booktitle=
-
[16]
IEEE/CVF International Conference on Computer Vision (ICCV) , year=
Segment anything , author=. IEEE/CVF International Conference on Computer Vision (ICCV) , year=
-
[17]
arXiv:2408.00714 , year=
Ravi, Nikhila and Gabeur, Valentin and Hu, Yuan-Ting and Hu, Ronghang and Ryali, Chaitanya and Ma, Tengyu and Khedr, Haitham and R. arXiv:2408.00714 , year=
-
[18]
International Conference on Machine Learning (ICML) , year=
On calibration of modern neural networks , author=. International Conference on Machine Learning (ICML) , year=
-
[19]
Conference on Empirical Methods in Natural Language Processing (EMNLP) , year=
Evaluating object hallucination in large vision-language models , author=. Conference on Empirical Methods in Natural Language Processing (EMNLP) , year=
-
[20]
Ding, Henghui and Liu, Chang and He, Shuting and Jiang, Xudong and Torr, Philip H. S. and Bai, Song , booktitle=
-
[21]
Kim, Moo Jin and Pertsch, Karl and Karamcheti, Siddharth and Xiao, Ted and Balakrishna, Ashwin and Nair, Suraj and Rafailov, Rafael and Foster, Ethan and Lam, Grace and Sanketi, Pannag and Vuong, Quan and Kollar, Thomas and Burchfiel, Benjamin and Tedrake, Russ and Sadigh, Dor...
-
[22]
Tao, Stone and Xiang, Fanbo and Shukla, Arth and Qin, Yuzhe and Hu, Xander and Yuan, Michael and Xie, Tongzhou and Huang, Zhiao and Liu, Xiao and others , journal=
-
[23]
and Guibas, Leonidas J
Xiang, Fanbo and Qin, Yuzhe and Mo, Kaichun and Xia, Yikuan and Zhu, Hao and Liu, Fangchen and Liu, Minghua and Jiang, Hanxiao and Yuan, Yifu and Wang, He and Yi, Li and Chang, Angel X. and Guibas, Leonidas J. and Su, Hao , booktitle=
Reviewed August 2, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.