{"id":"a6bcb2e9-4a99-417b-97cc-35c9c46ffaa5","arxiv_id":"2506.21005","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":14,"one_line_summary":"A two-module post-processing framework (tracking-based relabeling plus virtual bounding box injection) reports +1.6% to +3.1% mAP@50 on helmet violation detection, but only on a self-annotated test set with fitted confidence offsets.","lead":"VisionGuard is a post-processing pipeline that adds tracking-based label smoothing and virtual bounding boxes to helmet-detection models, reporting small mAP gains on the AI City Challenge data. The gains are measured on the authors' own re-annotated test set with hand-tuned confidence thresholds, so the headline improvement is not independently verifiable.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported 3.1% mAP gain is internally inconsistent: Section IV-C attributes +3.1% to Co-DETR while Table II shows +1.6%, making the central claim unreliable independent of annotation quality.","rationale":"The reader's REJECT verdict rests primarily on the lack of public ground truth and the manual re-annotation of test videos. That is a serious verification concern: every reported mAP depends on a private, unaudited annotation set, and no inter-annotator agreement or independent test-server evaluation is provided. However, the stress-test pass identifies an even more fundamental and objective problem that does not depend on annotation quality: the paper's own text and table directly contradict each other about which detector receives the 3.1% improvement. This internal inconsistency alone is sufficient to reject the central claim as written, because the abstract's headline number cannot be trusted. The component analysis further weakens the claim: Adaptive Labeling contributes only +0.001%, so virtually all of the reported gain comes from the Contextual Expander, which injects synthetic boxes at hand-tuned low confidences and is explicitly designed to push rare-class predictions into the top-100 mAP ranking (Section III-C, Algorithm 1). This is not a learned, generalizable detection improvement; it is a heuristic that is sensitive to the confidence offsets and the private test annotation schema. In good faith, the proposed pipeline is not entirely without merit—temporal label consistency and context-based recall are legitimate ideas—but as presented, the empirical evidence is both self-contradictory and unverifiable. The reader's verdict of REJECT therefore stands, and the additional internal inconsistency reinforces it. I mark agreement as 'partial' because the reader focused on annotation bias while the most load-bearing issue is the directly contradictory result reporting; both are valid, but the contradiction is a cleaner, self-contained ground for rejection.","tokens_in":7824,"tokens_out":3375,"duration_ms":37419,"concrete_test":"Re-run the ablation in Section IV-C from the authors' raw detector outputs and the released implementation of Algorithm 1, if available, and recompute mAP@50 for both detectors with all components enabled. If Co-DETR yields 44.945 and DETR yields 42.760, then Section IV-C's text is wrong and the abstract's '3.1% overall' is unsupported. If instead Co-DETR yields a 3.1% gain, Table II is wrong. Either outcome confirms the internal inconsistency; a null result (both text and table match one another and the reproduced numbers) would require the authors to clarify which detector achieves which gain.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim—'VisionGuard improves overall mAP by 3.1% compared to baseline detectors' (Abstract)—is contradicted by the paper's own ablation table and text. Table II reports Co-DETR baseline 44.221 → 44.945 (+1.6%) and DETR baseline 41.473 → 42.760 (+3.1%). However, Section IV-C states: 'The full method improves mAP@50 by +3.1% on Co-DETR and +1.6% on DETR.' This is a direct swap. If Table II is correct, the abstract overstates the gain by applying the DETR-only figure to 'baseline detectors'; if Section IV-C is correct, Table II is misreported. Either way, the headline number is not reliably supported. The only component that produces any gain is the Contextual Expander (AL alone gives +0.001% on both models); the Expander is a hand-crafted post-hoc injection of virtual boxes with confidence offsets (Algorithm 1, lines 6, 12–14) tuned to the evaluation protocol, not a learned detector improvement. Thus the 3.1% is both internally inconsistent and largely an artifact of a manually calibrated top-100 ranking manipulation, rather than a robust detector gain. The reader's concern about private manual re-annotation is valid, but this internal contradiction is a more immediate, objective flaw: even taking the authors' annotations at face value, the reported results are mutually incompatible.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"VisionGuard is a post-processing framework for helmet-violation detection. It combines Adaptive Labeling, which uses OC-SORT tracks to stabilize per-frame class labels through confidence-weighted voting and relabeling, and a Contextual Expander, which injects virtual bounding boxes with class-dependent confidence offsets to improve recall of rare rider-position classes. The authors evaluate on the AI City Challenge 2023/2024 test set, manually re-annotated with the nine-class schema, and report mAP@50 ablation results for DETR and Co-DETR baselines. The paper's headline claim is a 3.1% mAP improvement over baseline detectors.","tokens_in":8172,"tokens_out":7136,"duration_ms":74540,"significance":"The motivation is practical and the modular design is plausible: temporal smoothing of labels is a reasonable way to address inconsistent frame-wise predictions, and the paper is clearly organized. However, the quantitative significance is not established. The headline gain is contradicted by the paper's own ablation table, all results depend on an unvalidated private re-annotation of test videos, and the only nonzero component is a hand-crafted virtual-box module explicitly designed to influence the top-100 ranking used by mAP. If the authors could provide an official evaluation or an independently validated benchmark, the framework could be of interest to the traffic-surveillance community, but the current evidence is insufficient.","major_comments":[{"comment":"The paper states in the Abstract and Introduction that VisionGuard improves overall mAP by 3.1% compared to baseline detectors, but Section IV-C says \"The full method improves mAP@50 by +3.1% on Co-DETR and +1.6% on DETR,\" whereas Table II reports Co-DETR 44.221 to 44.945 (+1.6%) and DETR 41.473 to 42.760 (+3.1%). These two statements cannot both be correct, and the 3.1% figure applies to DETR only even if Table II is taken at face value. The central claim as stated is therefore unsupported by the paper's own data.","section":"Abstract; Section IV-C; Table II"},{"comment":"All reported mAP numbers are computed against ground-truth annotations that the authors manually re-annotated for the test videos (\"As ground-truth annotations for the test videos were not publicly provided, we manually re-annotated them\"). No inter-annotator agreement, no validation split, no error bars, and no official test-server evaluation are provided. Because the Contextual Expander's confidence offsets and all per-class AP values in Tables II and III depend on this private annotation set, the quantitative results cannot be verified or reproduced by an independent reader.","section":"Section IV-B"},{"comment":"The Contextual Expander is explicitly designed to increase the rank of rare-class detections: Section III-C states that class-dependent confidence adjustments \"increases the likelihood that these instances exceed the confidence threshold and are included among the top 100 detections per frame.\" Since mAP@50 is computed on the top-100 ranked predictions, adding virtual boxes with hand-fixed confidences (Algorithm 1, lines 6, 12-14) can change mAP without reflecting any learned improvement in detection. Table II shows that Adaptive Labeling alone contributes only +0.001% on both detectors, so the entire reported gain comes from this post-hoc ranking adjustment rather than from the proposed synergistic framework.","section":"Section III-C; Algorithm 1"},{"comment":"The ablation study provides no support for the word \"synergistic\": AL alone has essentially no effect (+0.001%), CE is never evaluated alone, and no sensitivity analysis is reported for the thresholds θq, θ0, α, λ, or for the OC-SORT parameters. It is therefore unknown whether the reported +1.6%/+3.1% is stable or an artifact of the specific parameter choices.","section":"Table II"}],"minor_comments":[{"comment":"The bullet \"We present the the Contextual Expander module\" contains a duplicated \"the.\"","section":"Section I, contribution list"},{"comment":"The adaptive threshold θ_i is not motivated: the factor (1 + (0.5 - Q_t)) is introduced without explanation, and no sensitivity analysis is provided for its effect on the relabeling behavior.","section":"Section III-B2, Eq. (3)"},{"comment":"The pseudocode's branch structure is confusing: line 15 creates a virtual motorbike from a human bounding box, while the default branch on line 25 also adds all human-related classes for every human; a textual explanation of the intended co-occurrence logic would improve reproducibility.","section":"Algorithm 1"},{"comment":"The module name is spelled inconsistently as \"Adaptive Labeling\" and \"Adaptive Labelling\"; please unify the spelling.","section":"Throughout"},{"comment":"Reference [3] is a 2024 master's thesis on sustainable commuting in Lahti and does not appear to support the claimed benefit of timely penalties for encouraging behavioral change.","section":"References"}],"recommendation":"reject","confidential_remarks":"The stress-test concern about internal inconsistency is confirmed by the manuscript: the abstract's 3.1% figure contradicts Section IV-C and Table II, and the evaluation rests on unvalidated private annotations. Even if the annotation issue were resolved, the Contextual Expander's explicit design around the top-100 mAP ranking means the reported gain is not evidence of improved detection. Rejection is appropriate."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper is a decently written post-processing recipe, but the headline number is not credible. The abstract says a 3.1% mAP gain over 'baseline detectors'; Table II shows that gain only for DETR, with Co-DETR at +1.6%; and Section IV-C claims the opposite swap. That internal contradiction alone makes the central claim unreliable, independent of annotation quality.\n\nCredit where due: the Adaptive Labeling module is a sensible extension of tracking-based label smoothing, and the Contextual Expander is spelled out as explicit pseudocode with all parameters listed. The authors also honestly note that Adaptive Labeling contributes essentially nothing (+0.001% mAP). That transparency is good.\n\nThe larger problem is what the Contextual Expander actually does. It injects synthetic boxes with tiny confidence offsets, and the offsets are hand-set to push rare classes into the top-100 ranking that mAP uses. That is fitting to the evaluation protocol, not improving the detector. The test set is manually re-annotated by the authors with no inter-annotator agreement or official server check, which makes every reported number unverifiable. No code, data, or error bars either.\n\nWho is this for? Someone working on post-processing for class-imbalanced detection in surveillance might find the recipe worth studying, but they should not trust the claimed gains. The paper is clear enough to be a useful baseline for that niche.\n\nRecommendation: I would not accept it as-is, but it deserves a serious referee rather than a desk reject. The method is concrete, the problem is real, and the flaws are fixable if the authors rerun on the official test server, report per-model gains consistently, and remove the internal contradiction.","headline":"A clearly written post-processing recipe whose headline 3.1% mAP gain is internally inconsistent and largely an artifact of tuning virtual boxes to mAP's top-100 protocol.","tokens_in":8759,"tokens_out":2494,"would_cite":false,"duration_ms":26634,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a two-stage post-processor—tracking-based label stabilization plus context-aware virtual boxes—raises helmet-violation detection mAP@50 by 3.1% on DETR and 1.6% on Co-DETR without retraining the detectors, and that…","keywords":["helmet violation detection","motorcycle safety","object detection","multi-object tracking","OC-SORT","class imbalance","traffic surveillance","virtual bounding boxes"],"falsifier":"Re-run both baseline detectors and the full VisionGuard on the same AI City Challenge test videos using the challenge organizers' official ground-truth labels, or a second independent annotation generated by different annotators, and compare mAP@50; if the 3.1% and 1.6% gains shrink or reverse, the result hinges on the private labels.","tokens_in":7585,"feed_emoji":"🏍️","tokens_out":7189,"duration_ms":72232,"temperature":0.7,"pith_summary":"VisionGuard is a post-processing framework for motorcycle helmet-violation detection. The authors claim that applying tracking-based Adaptive Labeling and a Contextual Expander to frame-wise transformer detectors raises overall mAP@50 by 3.1% on DETR and 1.6% on Co-DETR, with almost the entire gain coming from the Contextual Expander. If this holds, traffic-surveillance systems could improve rare-class recall and classification consistency without retraining the underlying detector, which matters because enforcement needs to catch uncommon rider positions such as P0 and P2 reliably.","feed_headline":"Helmet-violation mAP up 3.1% with virtual boxes","feed_subtitle":"Tracking-based labels and synthetic boxes for rare riders improve transformer detectors without retraining.","key_machinery":"Two coupled post-processing devices carry the argument. First, tracking-based Adaptive Labeling: OC-SORT assigns IDs, each track $t$ gets quality $Q_t=(1-r)\\bar{c}_t$ where $r$ is the label-change ratio and $\\bar{c}_t$ the mean confidence; tracks with $Q_t \\geq \\theta_q$ receive a label by confidence-weighted voting, and mislabeled detections are relabeled or removed using the adaptive threshold $\\theta_i = (\\theta_0 + \\alpha(1-\\bar{c}_t))(1 + (0.5-Q_t))$ and a confidence penalty $\\lambda$. Second, Contextual Expander: for every motorbike and human detection it emits synthetic boxes for correlated classes at the same location, scales the driver box to 70% for P0, uses tiny confidences ($10^{-5}$ or $10^{-4}$), and adds small offsets for rare classes so they survive the top-100 cutoff. The expansion mechanism is the load-bearing part: the ablation table attributes essentially all of the measured gain to it.","core_discovery":"The paper's central claim is that the weak spot in helmet-violation detection is not only the detector architecture but the way its per-frame outputs are ranked and labeled, and that a post-processing layer can fix much of it. Adaptive Labeling uses OC-SORT tracks, a track quality score $Q_t=(1-r)\\bar{c}_t$, and confidence-weighted voting to relabel or delete unstable detections. Contextual Expander inserts virtual bounding boxes for classes that rarely appear, assigns tiny confidences with class-specific additive offsets, and thereby pushes underrepresented riders into the top-100 detections scored by mAP. On the authors' re-annotated AI City Challenge test set, the full framework gives DETR mAP@50 of 42.760 (from 41.473) and Co-DETR 44.945 (from 44.221); the ablation shows Adaptive Labeling alone changes mAP by only 0.001%, so the measured gain is almost entirely the Contextual Expander's.","pith_inferences":["Beyond the paper: because the gain is almost entirely from synthetic boxes and top-100 ranking, a natural stress test is replacing Contextual Expander with random low-confidence boxes; if random boxes perform as well, the apparent gain is about rank-threshold effects rather than learned spatial context.","Beyond the paper: the entire evaluation rests on the authors' manual re-annotation, so the result should be re-checked on official challenge ground truth or a second independent annotation before relying on the 3.1% number.","Beyond the paper: the Adaptive Labeling module's 0.001% effect may be due to high camera angles and close rider spacing hurting track quality; in settings where riders are more separated or cameras are lower, tracking-based label correction could contribute more than the ablation here shows."],"forward_implications":["Deploying VisionGuard on an existing detector requires no retraining or new architecture; only tracking and box injection need to be added to the inference pipeline.","Rare rider classes that score 0.000 AP on the base detectors, such as P0NoHelmet and P2NoHelmet, register non-zero AP after the Contextual Expander, so enforcement systems can begin detecting previously invisible violations.","Because mAP@50 counts only the top-100 ranked predictions per frame, any post-processor that raises the rank of rare-class boxes can raise mAP even if localization is unchanged.","The same two-module recipe improves both an end-to-end DETR and an ensembled Co-DETR, so the framework is not tied to one detector family."],"supporting_citations":[{"why":"Supplies the OC-SORT tracker that builds temporally consistent tracks for Adaptive Labeling.","marker":"[11]"},{"why":"Provides the AI City Challenge training set and the test videos that the authors manually re-annotate.","marker":"[12]"},{"why":"DETR is one of the two baseline detectors whose raw outputs VisionGuard post-processes.","marker":"[8]"},{"why":"Co-DETR is the other baseline detector and the source of the ensembled checkpoints used in the Co-DETR pipeline.","marker":"[13]"},{"why":"Provides the Co-DETR ensemble recipe and the minority-class enhancement idea that inspires the Contextual Expander.","marker":"[24]"},{"why":"Supplies the confidence-adjustment strategy for underrepresented classes that the Contextual Expander adapts.","marker":"[27]"},{"why":"Weighted Box Fusion merges the multiple Co-DETR checkpoints before VisionGuard runs.","marker":"[28]"}],"fun_headline_variants":["Virtual boxes boost helmet-detection mAP by 3.1%","Tracking labels plus synthetic boxes lift mAP 3.1%","For rare riders, virtual boxes fix detection ranking","Post-processing adds virtual boxes to catch helmetless riders","Contextual Expander drives 3.1% mAP gain in helmet detection"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Every reported mAP and every confidence offset in the Contextual Expander is tuned against the authors' private manual re-annotation of the test videos; if that annotation is biased or error-prone, the claimed gains are not a property of the framework but of the labels.","fun_headline_variants_meta":{"raw":{"variants":["Virtual boxes boost helmet-detection mAP by 3.1%","Tracking labels plus synthetic boxes lift mAP 3.1%","For rare riders, virtual boxes fix detection ranking","Post-processing adds virtual boxes to catch helmetless riders","Contextual Expander drives 3.1% mAP gain in helmet detection"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.0002,"raw_usage":{"total_tokens":1369,"prompt_tokens":933,"completion_tokens":436,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":549,"completion_tokens_details":{"reasoning_tokens":348}},"tokens_in":549,"tokens_out":436,"duration_ms":5076,"temperature":1.0,"reasoning_tokens":348,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T22:36:13.900597+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run both baseline detectors and the full VisionGuard on the same AI City Challenge test videos using the challenge organizers' official ground-truth labels, or a second independent annotation generated by different annotators, and compare mAP@50; if the 3.1% and 1.6% gains shrink or reverse, the result hinges on the private labels.","supporting_citations":[{"cited_title":"The 7th ai city challenge,","cited_arxiv_id":null,"evidence_quote":"Provides the AI City Challenge training set and the test videos that the authors manually re-annotate."},{"cited_title":"End-to-end object de- tection with transformers,","cited_arxiv_id":null,"evidence_quote":"DETR is one of the two baseline detectors whose raw outputs VisionGuard post-processes."},{"cited_title":"Detrs with collaborative hybrid assignments training,","cited_arxiv_id":null,"evidence_quote":"Co-DETR is the other baseline detector and the source of the ensembled checkpoints used in the Co-DETR pipeline."},{"cited_title":"Robust motorcycle helmet detection in real-world scenarios: Using co-detr and minority class enhancement,","cited_arxiv_id":null,"evidence_quote":"Provides the Co-DETR ensemble recipe and the minority-class enhancement idea that inspires the Contextual Expander."},{"cited_title":"Motorcyclist helmet violation detection framework by leveraging robust ensemble and augmentation methods,","cited_arxiv_id":null,"evidence_quote":"Supplies the confidence-adjustment strategy for underrepresented classes that the Contextual Expander adapts."},{"cited_title":"Weighted boxes fusion: Ensembling boxes from different ob- ject detection models,","cited_arxiv_id":null,"evidence_quote":"Weighted Box Fusion merges the multiple Co-DETR checkpoints before VisionGuard runs."}],"review_version":1}