{"id":"a128b2c5-898d-4503-be88-86dee00143b1","arxiv_id":"2606.10666","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":4.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"Adapting training-free feature-space methods detects semantic mislabels reliably but not positional errors in object detection annotations on VOC2012 and KITTI.","lead":"The paper adapts feature-space methods to detect annotation errors in object detection datasets. It finds these methods reliably catch semantic mislabels but struggle with positional errors across models and datasets.","discovery_kind":"new_application","skeptic_critique":{"model":"grok-4.3","headline":"No significant objection identified","rationale":"Reader's weakest assumption is the separability premise independent of model/dataset. The manuscript addresses this by explicit multi-model, multi-dataset evaluation on both synthetic and real errors, removing the load-bearing risk. No other internal inconsistency or unsupported step is visible in the central claim.","tokens_in":1664,"tokens_out":237,"duration_ms":10202,"concrete_test":"Re-run the semantic vs. positional error detection comparison on the publicly released code using one additional embedding model (e.g., a ResNet-50 variant not already tested) and confirm that the reported gap in detection reliability remains above the threshold shown in the paper's tables.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper evaluates the adapted feature-space method across multiple pretrained embedding models, three synthetic noise types (symmetric, asymmetric, positional), and real-world annotation errors on VOC2012 and KITTI. This directly tests the separability of semantic mislabels from positional errors in embedding space and reports consistent differential behavior, satisfying the condition that the result not depend on a single model or dataset choice.","agreement_with_reader":"disagree"},"referee_report":{"model":"grok-4.3","summary":"The manuscript adapts an existing training-free feature-space method to analyze annotation errors in object detection datasets. It claims that such methods reliably detect semantic mislabels while positional errors remain difficult to detect. The evaluation covers multiple pretrained embedding models, synthetic noise types (symmetric, asymmetric, positional), and real-world errors on VOC2012 and KITTI, with code and real-world corruptions released publicly.","tokens_in":1704,"tokens_out":172,"duration_ms":13443,"significance":"If the empirical findings hold, the work clarifies the differential behavior of embedding-space distances for semantic versus positional annotation errors in object detection, with consistency across models and datasets strengthening the result. The public release of code and corruptions supports reproducibility and is a clear strength.","major_comments":[],"minor_comments":[],"recommendation":"accept","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for their positive assessment of the manuscript and for recommending acceptance. We appreciate the recognition of the empirical findings on semantic versus positional errors, the consistency across models and datasets, and the value of the public code and corruption release.","responses":[],"tokens_in":1131,"tokens_out":67,"duration_ms":5438,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main point is that adapting an existing feature-space method to object detection shows it reliably flags semantic label errors while positional errors stay hard to catch. The authors test this across multiple pretrained embeddings, three synthetic noise types, and real annotation issues on VOC2012 and KITTI.\n\nThey release the code and the real corruption data, which makes the work easy to check. Running the same approach on different models and both synthetic and actual errors addresses the usual concern that results might be tied to one embedding or one dataset. That setup is a clear strength for an empirical analysis.\n\nThe soft spot is that the abstract gives no numbers, so the size of the gap between error types is unclear without the full paper. If the results include clear metrics and controls for embedding choice, the differential behavior looks solid; otherwise it stays mostly descriptive. No load-bearing assumptions seem violated based on the described experiments.\n\nThis is for people cleaning object detection datasets who need quick checks on annotation quality. It does not introduce new methods but maps where an existing class of tools works and where it does not. The experiments are reproducible enough that it deserves peer review to verify the details and see if the conclusions hold.","headline":"Feature-space outlier detection catches semantic mislabels in object detection annotations but not positional errors, with tests across models and datasets plus public code.","tokens_in":2167,"tokens_out":313,"would_cite":false,"duration_ms":14822,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Feature-space distances reliably flag semantic label errors in object detection datasets but leave positional errors hard to detect.","keywords":["annotation errors","object detection","feature space","semantic mislabels","positional errors","training-free detection","VOC2012","KITTI"],"falsifier":"A single pretrained embedding model paired with one of the tested datasets in which feature distances separate positional errors at least as well as semantic errors, or fail to separate semantic errors at all.","tokens_in":2562,"feed_emoji":"🔍","tokens_out":612,"duration_ms":13363,"temperature":0.7,"pith_summary":"The paper examines whether training-free methods based on distances in pretrained feature embeddings can identify annotation mistakes in object detection data. It adapts an existing technique and runs controlled tests with synthetic symmetric, asymmetric, and positional noise plus real errors drawn from VOC2012 and KITTI. Semantic mislabels are consistently exposed while bounding-box position mistakes remain difficult to separate from correct annotations. The pattern holds across several embedding models. This matters because manual review of large detection datasets is costly, so any automatic filter that catches one major error type without training could reduce cleaning effort.","feed_headline":"Feature embeddings flag semantic label errors but miss position mistakes","feed_subtitle":"Adapting training-free methods on VOC2012 and KITTI shows reliable detection of mislabels across models while positional corruptions remain","key_machinery":"Distances computed in the feature space of a pretrained embedding model applied to image crops of detected objects.","core_discovery":"By adapting an existing feature-space method, we show that such approaches reliably expose semantic mislabel, while positional errors remain difficult to detect. We evaluate this behavior across multiple pretrained embedding models, synthetic noise types (symmetric, asymmetric, and positional), and real-world annotation errors using VOC2012 and KITTI.","pith_inferences":["Dataset pipelines could run embedding checks first to triage label errors before investing in positional review.","Combining the feature-space filter with geometric heuristics for box placement would address the remaining error class.","The observed independence from model choice hints that the approach may transfer to other detection datasets without retuning.","Large-scale annotation projects could insert this step to lower the fraction of annotations needing human inspection."],"forward_implications":["Semantic mislabels can be surfaced without training a dedicated detector.","Positional annotation errors require separate detection strategies.","The separation pattern is stable across the embedding models examined.","Both synthetic noise and real annotation mistakes in VOC2012 and KITTI exhibit the same split in detectability."],"fun_headline_variants":["Embeddings flag semantic mislabels but miss position errors in detection","Feature methods reliably detect label errors but not spatial corruptions","Semantic errors exposed positional ones difficult to find across models","Training free approaches catch mislabels positional errors remain challenging"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"Distances in pretrained embedding space separate semantic annotation errors from correct ones while failing to separate positional errors, and this separation does not depend on the choice of embedding model or dataset.","fun_headline_variants_meta":{"raw":{"variants":["Embeddings flag semantic mislabels but miss position errors in detection","Feature methods reliably detect label errors but not spatial corruptions","Semantic errors exposed positional ones difficult to find across models","Training free approaches catch mislabels positional errors remain challenging"]},"model":"grok-4.3","cost_usd":0.006232,"raw_usage":{"total_tokens":2903,"prompt_tokens":605,"num_sources_used":0,"completion_tokens":64,"cost_in_usd_ticks":62324500,"prompt_tokens_details":{"text_tokens":605,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":2234,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":605,"tokens_out":64,"duration_ms":13221,"temperature":1.0,"reasoning_tokens":2234,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-27T13:45:05.369419+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"A single pretrained embedding model paired with one of the tested datasets in which feature distances separate positional errors at least as well as semantic errors, or fail to separate semantic errors at all.","supporting_citations":[],"review_version":1}