{"id":"1d57f568-eb6f-47c7-bcd5-a0a9b6c9830c","arxiv_id":"2412.20701","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"The paper modifies Faster R-CNN with CLIP-based semantic alignment, feature decorrelation, and centerness-based objectness to improve open-set detection, reporting higher unknown-class precision on VOC-COCO.","lead":"This paper proposes an open-set object detector that flags objects from unseen classes as 'unknown'. It combines semantic clustering with CLIP text embeddings, a class decorrelation loss, and an object focus loss, reporting gains on VOC and COCO benchmarks.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper's claimed SOTA improvements likely stem from an asymmetric evaluation: entropy thresholding (threshold 0.85 tuned on the test set) is applied to the proposed model, but the paper never states that baselines receive the same post-processing, so APu/WI/AOSE gains may be evaluation artifacts.","rationale":"I read the paper in good faith: the architectural components (semantic clustering, class decorrelation, object focus loss) are plausible and the ablations in Table 3 suggest each contributes. However, the central claim is about beating prior SOTA on open-set metrics, and that comparison is only valid if the evaluation protocol is identical across methods. The paper introduces entropy thresholding as a contribution and tunes the threshold on the test set, yet never states that baselines receive the same treatment. Since this post-processing mechanically converts low-confidence known predictions into 'unknown' detections, it directly improves every unknown-object metric reported. The absence of a baseline-with-threshold control is not a minor omission; it undermines all headline comparisons. The reader's weakest-assumption analysis correctly identifies this, so I agree with the verdict. A single re-evaluation experiment would settle whether the effect is real. If the advantage survives the control, the method may indeed be valuable; if not, the paper requires major revision before its claims can be trusted.","tokens_in":17200,"tokens_out":7439,"duration_ms":70809,"concrete_test":"Re-evaluate all baselines (Faster R-CNN, ORE, DS, PROSER, OpenDet, Openset RCNN) on VOC-COCO-T1 and T2 using the paper's entropy thresholding rule: after obtaining each method's classification logits, recompute the known-class scores and relabel any proposal whose softmax entropy exceeds 0.85 as 'unknown', then recompute WI, AOSE, APu, and HMP. Also report the proposed method without this post-processing. If baselines' APu/WI/AOSE improve to near the proposed method's levels, or the proposed method's advantage shrinks below the reported 17-24%, the headline improvement is an evaluation artifact; if the advantage persists, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing weakness is the evaluation protocol around 'Entropy Thresholding' (Section 4.1). The paper defines a post-processing rule: if the entropy of the classification logits exceeds 0.85, the proposal is relabeled as 'unknown'. The threshold is selected on the VOC-COCO-40 test set (Supplementary B.2, Figure 7) and applied to all reported results of the proposed model. However, the comparison tables (Tables 1, 2, 5-7) give no indication that the same relabeling is applied to baseline detectors. Because relabeling low-confidence known-class predictions as 'unknown' directly lowers WI and AOSE and raises APu (and therefore HMP), the claimed APu improvements of 17-24% over OpenDet (Section 4.2) may be largely attributable to this asymmetric post-processing rather than to the model's learned representations. The mAPk drops observed for the proposed model in several settings (e.g., 73.47 vs 75.56 for OpenDet in Table 2, VOC-COCO-n) are consistent with this: the threshold trades known-class precision for unknown-class recall. The paper never reports an ablation of entropy thresholding on the proposed model or a control with baselines under the same threshold, so the reader's concern is a genuine, testable threat to the central claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an open-set object detection (OSOD) framework built on Faster R-CNN. The contributions are: (i) a semantic clustering module that aligns proposal features with CLIP text embeddings of known classes, (ii) a class decorrelation module that imposes an orthogonality constraint on sampled features, (iii) an object focus loss combining centerness and classification-based objectness for the RPN, and (iv) an evaluation technique called entropy thresholding that relabels low-confidence predictions as unknown. The authors also introduce the Harmonic Mean Precision (HMP) metric, defined as the harmonic mean of mAPk and APu. Experiments on PASCAL VOC and MS-COCO under the VOC-COCO-T1 and T2 protocols report consistent improvements over OpenDet and other baselines, with APu gains of 17–24% on ResNet50 in the T1 setting, along with reduced WI and AOSE.","tokens_in":17490,"tokens_out":5052,"duration_ms":49173,"significance":"If the reported results are valid, the paper would be a useful contribution to OSOD: the semantic clustering idea is reasonable, the ablation studies are thorough, and the HMP metric, though simple, provides a convenient single-number summary. The authors also provide extensive comparisons across three backbones and multiple open-set settings, and they include failure-case analysis. However, the significance rests on the validity of the evaluation protocol. The entropy-thresholding post-processing is a potential source of asymmetric evaluation, and the threshold is tuned on the test set, which is a data-leakage concern. These issues are load-bearing because the claimed improvements over prior work are the central contribution.","major_comments":[{"comment":"The entropy threshold is set to 0.85 based on 'empirical analysis' on the VOC-COCO-40 test set (Supplementary Figure 7). Selecting a post-processing parameter on the test set and then reporting test-set results with that parameter constitutes test-set leakage, which inflates the reported performance and prevents a fair comparison with baselines. The threshold should be chosen on a held-out validation split, and the sensitivity of the main result to the threshold value should be reported. As it stands, the central claim of significant improvement is not supported by a clean evaluation protocol.","section":"Supplementary B.2 and Section 4.1"},{"comment":"The class decorrelation loss is computed on a single sampled feature per unique class per batch, and the cross-entropy with an identity matrix encourages the sampled features to be orthogonal. The paper claims this 'enhances inter-cluster distance' and 'separates the clusters,' but the loss only constrains one feature per class and does not guarantee orthogonality of all features within a cluster. The theoretical justification is therefore weak, and the small improvement observed in the ablation (Case 4 vs. Case 1 in Table 3) does not clearly establish the mechanism. Please clarify whether the constraint is intended as an approximation, and provide evidence (analytical or empirical) that it affects the entire feature distribution, or temper the claim accordingly.","section":"Section 3.3, Eq. (5)"}],"minor_comments":[{"comment":"The denominator of the softmax uses the symbol 'n' without definition; it should be 'k' (the number of classes), consistent with Eq. (4).","section":"Section 3.2, Eq. (2)"},{"comment":"References [19] and [20] are the same paper (Sun et al., 'Fsce: Few-shot object detection via contrastive proposal encoding'), and references [22] and [23] are also the same paper (Wu et al., 'Two-branch objectness-centric open world detection'). Please deduplicate.","section":"References"},{"comment":"HMP is introduced as a new metric, but it is simply the harmonic mean of mAPk and APu. Please clarify whether this is intended as a contribution or as a reporting convenience, and compare it with existing combined metrics if one exists.","section":"Section 4.1, HMP metric"},{"comment":"The paper does not mention code release or provide training/evaluation scripts, which limits reproducibility. Consider adding a reproducibility statement or a link to code.","section":"Implementation Details"}],"recommendation":"major_revision","confidential_remarks":"The entropy-thresholding concern is central and, in my view, valid. If the authors can re-run the evaluation with symmetric post-processing and a properly chosen threshold (on validation data), the paper may become acceptable. However, if the reported gains evaporate under those conditions, the contribution would be substantially weaker. I would advise the editor to require the re-evaluation before any acceptance decision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the paper is a sensible combination of known OSOD ingredients, and the ablations are decent, but the headline numbers are not trustworthy because the entropy-threshold post-processing is applied to the proposed model and apparently not to the baselines. That makes the claimed SOTA gains mostly uninterpretable.\n\nWhat's genuinely new: the triple combination of CLIP-based semantic clustering, feature decorrelation, and the centerness-weighted object focus loss is not in the prior literature, and the HMP metric, while trivial, is a reasonable way to summarize known/unknown trade-offs. The ablations in Table 3 show each module contributes, and Table 4 supports the geometric-mean choice. The paper also includes failure cases and a threshold-sensitivity plot, which is more than many papers do.\n\nThe soft spot: Section 4.1 introduces entropy thresholding as an evaluation technique with threshold 0.85, chosen on VOC-COCO-40 (Supplementary B.2). It relabels any proposal whose classification entropy exceeds 0.85 as 'unknown.' The comparison tables never say whether baseline detectors get the same post-processing, and nothing in the text suggests they do. That matters because relabeling low-confidence known-class predictions directly reduces WI and AOSE and can inflate APu, at the cost of known-class mAPk. The mAPk drops the proposed model shows relative to OpenDet in several rows (73.47 vs 75.56, 69.44 vs 71.44, etc.) are exactly the signature of this trade-off. Without a control that applies the same threshold to baselines, the 17-24% APu improvement claim is not supported. This is a fixable problem: redo the evaluation with the same threshold for everyone, or report results without thresholding. But it is load-bearing in the current version.\n\nVerdict: worth a serious referee but not acceptable as-is. A reviewer could reasonably request the re-evaluation, and the method might survive it. I'd send it out.","headline":"Sensible OSOD combination with a decent ablation story, but the headline gains are uninterpretable because the entropy-threshold post-processing is applied to the proposed model and apparently not to the baselines.","tokens_in":18013,"tokens_out":1717,"would_cite":false,"duration_ms":17562,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that aligning known-class proposals to semantic text embeddings, decorrelating clusters, and learning objectness from centerness lets an open-set detector correctly mark unseen objects as unknown instead of…","keywords":["open-set object detection","semantic clustering","class decorrelation","object focus loss","entropy thresholding","harmonic mean precision"],"falsifier":"Run the OpenDet baseline with the same entropy-thresholding post-processing at 0.85 on VOC-COCO-40; if its APu rises to near the proposed model's 14.00, the reported gains are mostly an artifact of the evaluation rule rather than the learned clusters. Alternatively, evaluate the proposed model with entropy thresholding disabled and compare APu to OpenDet without it.","tokens_in":16992,"feed_emoji":"🎯","tokens_out":5614,"duration_ms":45158,"temperature":0.7,"pith_summary":"The paper tackles open-set object detection: finding objects from classes never seen during training and marking them 'unknown' rather than forcing them into a known category. Its central claim is that prior contrastive-clustering detectors fail on unknowns that are semantically close to a known class (a zebra called a horse) because their clusters are not aligned to semantic meaning. The proposed fix aligns region-proposal features with CLIP text embeddings of class names, enforces orthogonality between class clusters, and learns objectness from centerness and geometry rather than from class labels alone. The authors report large gains in unknown-detection precision (APu) and lower misclassification (AOSE, WI) on MS-COCO and PASCAL VOC, and introduce a combined metric, harmonic mean precision, to summarize known-plus-unknown performance.","feed_headline":"Semantic clusters stop unknown objects being mislabeled","feed_subtitle":"Proposal features aligned to CLIP text embeddings plus decorrelation cut unknown-class errors on VOC-COCO.","key_machinery":"The load-bearing mechanism is the semantic clustering module: a cross-entropy loss that pulls each ROI-aligned proposal feature toward the CLIP text embedding of its ground-truth class name, so clusters form around semantic axes (Eq. 1–2). Around it sit two supporting modules: a class decorrelation loss that samples one feature per class per batch and diagonalizes their cosine-similarity matrix via a softmax cross-entropy term (Eq. 3–5), and an object focus loss that replaces pure classification-based objectness with the geometric mean of a centerness loss and the standard objectness loss (Eq. 6–7), so the RPN learns generic 'thingness' rather than memorizing training categories. At evaluation, an entropy threshold (0.85) relabels low-confidence known-class predictions as 'unknown', and a new metric, HMP, is defined as the harmonic mean of known mAP and unknown APu.","core_discovery":"In the authors' telling, the reason open-set detectors overconfidently mislabel unknowns is that their known-class feature clusters are arranged by instance-level contrastive learning, which does not respect semantic class boundaries. The paper's discovery is that replacing that implicit clustering with an explicit semantic alignment—cosine-similarity cross-entropy between proposal features and CLIP text embeddings of the class names—together with a softmax-form orthogonality constraint on sampled class features and a geometric-mean object focus loss, yields well-separated clusters that leave room for unknown objects to be detected as unknowns. On the VOC-COCO-T1 and T2 protocols with ResNet50, ConvNet, and Swin-T backbones, the proposed model reports APu improvements of roughly 4–18 points over the OpenDet baseline and consistent reductions in Wilderness Impact and Absolute Open-Set Error, with a modest known-class mAP trade-off.","pith_inferences":["Because the threshold 0.85 was selected on the VOC-COCO-40 test set (Supplementary B.2), the absolute APu numbers are optimistic; a fair cross-method comparison should apply the same entropy rule to baselines or tune thresholds on a held-out split. This is our editorial caution, not the paper's claim.","The decorrelation module samples one feature per class per batch; in large-batch settings this may underrepresent intra-class variance, so extending it to multiple features per class with a diversity-aware sampler could further separate clusters—testable but beyond the paper.","The reported failure cases (false-positive 'unknown' detections) suggest the object focus loss increases recall at the cost of precision; a precision-recall trade-off study varying the entropy threshold would quantify this.","Since CLIP embeddings are frozen and trained on web-scale image-text data, the semantic axes may be biased toward the 80 COCO/VOC class names; evaluating on classes outside that distribution would test whether the alignment generalizes."],"forward_implications":["If the alignment claim holds, open-set detectors trained on the same closed-set data can be made markedly safer for robotics and autonomous driving, where mislabeling a novel obstacle as a known class is more dangerous than saying 'unknown'.","The entropy-thresholding evaluation rule, applied post-hoc, turns any detector into a better open-set detector; the paper's reported gains should be understood as combining model improvements with this evaluation change.","The HMP metric gives the community a single number that punishes detectors which ignore unknowns (APu=0 gives HMP=0), so future OSOD papers can compare trade-offs on one axis.","Semantic alignment via frozen text embeddings may transfer to other open-set tasks like incremental detection and open-set domain adaptation, as the authors note.","The known-class mAP cost is small but nonzero in several settings, suggesting the approach trades a little closed-set accuracy for open-set safety."],"supporting_citations":[{"why":"OpenDet is the primary contrastive-clustering baseline the proposed method compares against and the source of the UPL loss for unknown probability.","marker":"[6]"},{"why":"CLIP text encoder supplies the frozen class-name embeddings that anchor the semantic clustering module.","marker":"[17]"},{"why":"Faster R-CNN is the base detector architecture and the source of the classification-based objectness loss used in the object focus loss.","marker":"[18]"},{"why":"The feature decorrelation idea and softmax-form orthogonality constraint in the class decorrelation module are adapted from this work.","marker":"[25]"},{"why":"The centerness target formulation for the object focus loss is taken from FCOS.","marker":"[21]"},{"why":"This work motivates the classification-free objectness/centerness approach that the object focus loss builds on.","marker":"[10]"},{"why":"PASCAL VOC is the closed-set training dataset and the source of the known classes in the open-set evaluation protocols.","marker":"[3]"},{"why":"MS-COCO supplies the unknown-class images and the 60 non-VOC classes used to build the VOC-COCO-T1 and T2 test sets.","marker":"[11]"},{"why":"ORE introduces the open-world detection protocol and the recall level at which Wilderness Impact is reported in this paper.","marker":"[8]"}],"fun_headline_variants":["Aligning known classes to CLIP text lifts unknown-object detection","Semantic alignment plus decorrelation finds unknown objects better","CLIP-aligned clusters improve open-set detection on VOC-COCO","New metric HMP and decorrelation cut unknown-class errors"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The evaluations assume that applying the entropy-thresholding relabeling (threshold 0.85, picked on the VOC-COCO-40 test set) only to the proposed model—while baselines are evaluated without it—does not inflate the reported improvements in APu, WI, and AOSE.","fun_headline_variants_meta":{"raw":{"variants":["Aligning known classes to CLIP text lifts unknown-object detection","Semantic alignment plus decorrelation finds unknown objects better","CLIP-aligned clusters improve open-set detection on VOC-COCO","New metric HMP and decorrelation cut unknown-class errors"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000714,"raw_usage":{"total_tokens":3172,"prompt_tokens":869,"completion_tokens":2303,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":485,"completion_tokens_details":{"reasoning_tokens":2234}},"tokens_in":485,"tokens_out":2303,"duration_ms":15937,"temperature":1.0,"reasoning_tokens":2234,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T23:12:22.004383+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the OpenDet baseline with the same entropy-thresholding post-processing at 0.85 on VOC-COCO-40; if its APu rises to near the proposed model's 14.00, the reported gains are mostly an artifact of the evaluation rule rather than the learned clusters. Alternatively, evaluate the proposed model with entropy thresholding disabled and compare APu to OpenDet without it.","supporting_citations":[{"cited_title":"Expanding low-density latent regions for open-set object detection","cited_arxiv_id":null,"evidence_quote":"OpenDet is the primary contrastive-clustering baseline the proposed method compares against and the source of the UPL loss for unknown probability."},{"cited_title":"Learning transferable visual models from natural language supervi- sion","cited_arxiv_id":null,"evidence_quote":"CLIP text encoder supplies the frozen class-name embeddings that anchor the semantic clustering module."},{"cited_title":"Faster r-cnn: Towards real-time object detection with region proposal networks","cited_arxiv_id":null,"evidence_quote":"Faster R-CNN is the base detector architecture and the source of the classification-based objectness loss used in the object focus loss."},{"cited_title":"Clustering- friendly representation learning via instance discrimination and feature decorrelation","cited_arxiv_id":null,"evidence_quote":"The feature decorrelation idea and softmax-form orthogonality constraint in the class decorrelation module are adapted from this work."},{"cited_title":"Fcos: Fully convolutional one-stage object detection","cited_arxiv_id":null,"evidence_quote":"The centerness target formulation for the object focus loss is taken from FCOS."},{"cited_title":"Learning open-world object proposals without learning to classify","cited_arxiv_id":null,"evidence_quote":"This work motivates the classification-free objectness/centerness approach that the object focus loss builds on."},{"cited_title":"The pascal visual object classes (voc) challenge","cited_arxiv_id":null,"evidence_quote":"PASCAL VOC is the closed-set training dataset and the source of the known classes in the open-set evaluation protocols."},{"cited_title":"Microsoft coco: Common objects in context","cited_arxiv_id":null,"evidence_quote":"MS-COCO supplies the unknown-class images and the 60 non-VOC classes used to build the VOC-COCO-T1 and T2 test sets."}],"review_version":1}