{"id":"e0d77564-b840-42d8-802d-90699aa3045c","arxiv_id":"2505.23726","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"FMG-Det adds an offline SAM plus CLIP box-correction stage and a learned noisy/corrected box interpolation to MIL-based detection, sharply reducing accuracy loss under heavy bounding-box noise on VOC and COCO, including few-shot settings.","lead":"FMG-Det trains object detectors on datasets whose bounding-box labels are noisy by first correcting every box with two foundation models, SAM and CLIP, then letting a small learned module blend each corrected box with the original. On the standard VOC and COCO benchmarks it retains far more accuracy at the highest noise levels than prior robust-detection methods, and it is the first test of this noise regime in few-shot detection.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 1 shows the noise-free baseline Faster R-CNN improving from 13.5 mAP at noise 0.8 to 19.0 mAP at noise 1.0, so the synthetic severity scale may be non-monotonic and the headline MAE gains are not yet on solid ground.","rationale":"The reader's weakest assumption concerns SAM/CLIP quality on real noisy data, which is a genuine external-validity limitation and is candidly acknowledged in Limitations F. My concern is different and more internal: the noise protocol itself appears not to be a monotonic severity scale, based on the baseline numbers in Table 1. This matters because every headline number, including the abstract's 6.8 and 6.6 MAE gains, is computed with respect to that protocol. If the protocol is non-monotonic, the comparison at high noise is not a controlled stress test, and the paper's main evidence base needs to be rebuilt even though the method itself may still be effective. The paper has real strengths: a clean ablation showing the FMC pipeline is responsible for most of the gain, error bars on the VOC sweep, and a careful limitations section. A simple protocol audit, including IoU diagnostics and multi-seed baseline training, is sufficient to settle the concern. If the audit confirms monotonicity, the original conditional verdict stands; if it does not, the reported numbers need correction before acceptance. For these reasons I recommend keeping the verdict CONDITIONAL, with the protocol audit added as an explicit condition.","tokens_in":11643,"tokens_out":10638,"duration_ms":107383,"concrete_test":"Reproduce the Section 3 noise generator exactly on VOC 2007 (and COCO if time permits), and for each level 0.0-1.0 compute the mean and median IoU between the noisy and original boxes over all training instances, along with the fraction of noisy boxes that are clipped or discarded because they fall outside the image. Then train the same Faster R-CNN baseline used in Table 1 with at least 3 seeds per level and verify that mAP is monotonically non-increasing in noise level. If IoU decreases monotonically but baseline mAP still rises at 1.0, inspect the generated boxes for boundary effects; if IoU itself is non-monotonic, the noise implementation is mislabeled and all MAE comparisons must be re-run under a corrected monotonic protocol before the headline gains can be trusted.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central empirical claim is that FMG-Det retains accuracy as annotation noise increases, summarized by MAE across noise levels 0.0-1.0. This requires the Section 3 noise injection to be monotonic in the noise parameter: level 1.0 must be strictly more damaging than level 0.8. Table 1 contradicts this requirement for the baseline detector: Faster R-CNN gets 13.5 +/- 0.12 mAP at 0.8 but 19.0 +/- 0.52 mAP at 1.0, a 5.5-point increase far outside the reported error bars; OA-MIL likewise rises from 16.2 +/- 0.25 to 18.2 +/- 0.40. If the noise protocol is monotonic, these entries should be non-increasing, and if the protocol is not monotonic, then the claimed \"severe noise\" comparisons and the MAE differences in the abstract are not a controlled stress test. One plausible mechanism is that at level 1.0 many shifted boxes fall outside the image and are clipped or discarded, so the effective number of corrupted training boxes can decrease rather than increase. The paper does not report the IoU distribution between noisy and clean boxes per level or the fraction of discarded boxes, so the severity ordering cannot be checked from the text. This is an internal correctness risk to the headline numbers rather than a dispute about external consensus.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes FMG-Det, a method for training object detectors when bounding-box annotations are noisy. It combines an offline Foundation Model Correction (FMC) pipeline, which uses SAM and CLIP to generate and select corrected boxes from noisy ones, with the OA-MIL multiple-instance-learning framework and a learned instance-interpolation module. The method is evaluated on PASCAL VOC and MS COCO under a synthetic bounding-box noise protocol, in both fully supervised and few-shot settings. The authors report large MAE reductions compared with Faster R-CNN, OA-MIL, and SSD-Det, and they claim average gains of 6.6 mAP on VOC and 6.8 mAP on COCO.","tokens_in":11760,"tokens_out":4984,"duration_ms":46316,"significance":"If the reported results hold, FMG-Det offers a simple, detector-agnostic pre-processing step that could substantially improve robustness to annotation noise, especially in few-shot settings, and the paper provides an honest ablation study and an explicit limitations section. The central weakness is that the strongest evidence is concentrated on the VOC full-training sweep, while the COCO evidence is thin and the synthetic severity scale appears non-monotonic in the reported numbers. Because the headline claims are framed through the MAE metric, the non-monotonicity directly affects whether the claimed gains are a controlled stress test.","major_comments":[{"comment":"The reported results are inconsistent with a monotone noise severity scale, which the MAE metric requires. For Faster R-CNN, mAP at noise 0.8 is 13.5 +/- 0.12 but at noise 1.0 it is 19.0 +/- 0.52; OA-MIL similarly rises from 16.2 +/- 0.25 to 18.2 +/- 0.40. Since the paper defines MAE as the average absolute drop relative to the noise-free baseline across levels, these entries imply that level 1.0 is not strictly more damaging than level 0.8. The authors should report per-level statistics such as the IoU distribution between noisy and clean boxes and the fraction of boxes clipped or discarded after perturbation, or otherwise justify the severity ordering; without this, the 'severe noise' comparisons and the MAE gains are not on solid ground.","section":"Section 3 and Table 1"},{"comment":"The COCO experiment reports MAE values but omits the noise-0.0 baseline mAP for every model and evaluates at only two noise levels. Because the abstract's 6.8-point COCO gain is based on this MAE, the central COCO claim cannot be checked from the paper. The authors should add the 0.0 column, error bars over runs, and ideally intermediate noise levels; the current two-point evaluation is too thin to support the headline comparison.","section":"Table 2 (MS COCO)"},{"comment":"The noise injection protocol is under-specified. The text says 'Values Δx, Δy, Δw, and Δh are sampled from a uniform distribution on the interval (−0.0, 1.0)' but does not state how the noise level parameter (0.0–1.0) maps to the sampling interval, nor whether coordinates are clipped or boxes discarded when they fall outside the image. This makes the experiments non-reproducible and is likely connected to the non-monotonicity reported in Table 1. A precise formula or pseudocode for the noise generation is needed.","section":"Section 3"},{"comment":"The abstract states an 'average performance gain of 6.8 mAP on COCO and 6.6 mAP on VOC', but Tables 1–2 report reductions in MAE (the average drop in mAP), not average mAP increases. These are different quantities, and the wording should be corrected to avoid claiming a gain in mean average precision that the experiments do not directly report.","section":"Abstract and Section 5.1.1"}],"minor_comments":[{"comment":"There are several typographical and formatting issues, including 'FOUNDA TION' in the title, 'VA' in the affiliation line, and 'stocastic' in the Limitations section; these should be cleaned up.","section":"General"},{"comment":"Table 2 reports no variance or error bars, unlike Table 1 which does; this makes it difficult to assess the reliability of the COCO results, especially the 15.7 mAP entry at noise 0.8.","section":"Table 2"},{"comment":"The instance interpolation module uses the detector's ROI extractor and backbone features, but it is not clear whether gradients flow through the ROI features and backbone during training of Gθ or only through the small MLP; this should be clarified.","section":"Section 4.2.1"},{"comment":"The few-shot results exhibit the same non-monotonic pattern as Table 1, e.g., 1-shot Faster R-CNN gets 0.4 mAP at noise 0.8 and 1.7 mAP at noise 1.0; a dedicated analysis of why the noise scale is not monotonic would strengthen the paper.","section":"Supplementary Table 3"},{"comment":"The λ = 0.05 IoU threshold for accepting a correction is very permissive and was selected 'empirically running on subsets of the data'; a sensitivity analysis over α and λ, or at least a statement of how the held-out test set was protected from this selection, would improve confidence in the method's hyperparameter choices.","section":"Section 4.1.3"},{"comment":"The paper references the OA-MIL and MMDetection repositories but does not provide code or a complete list of few-shot seeds; releasing the code and seed details would make the few-shot results reproducible.","section":"Reproducibility"}],"recommendation":"major_revision","confidential_remarks":"The non-monotonicity in Table 1 is the key risk: if it stems from clipping/discarding boxes at high noise, the MAE metric may be measuring a mix of damage and box removal rather than a controlled severity sweep. The COCO evidence is also thinner than the abstract suggests. These are fixable within the scope of the paper if the authors provide the missing protocol details and per-level diagnostics; otherwise the headline claims should be substantially qualified."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe useful thing about FMG-Det is that it is a genuinely simple, detector-agnostic fix: run a SAM/CLIP correction pass offline, then interpolate between corrected and noisy boxes with a small learned module inside an OA-MIL detector. The VOC sweep is the best evidence, and the authors deserve credit for an honest ablation — FMC alone gets MAE 17.5, adding OA-MIL on top makes it 18.0, and they say so. The limitations section is unusually candid about SAM failure and the lack of a naturally noisy benchmark.\n\nThe soft spot is real, though. The stress-test check is not a nitpick: Table 1 has Faster R-CNN at 13.5 mAP on noise 0.8 and 19.0 on noise 1.0, and OA-MIL likewise rises from 16.2 to 18.2. If the noise level is supposed to be monotonic, these numbers should not go up. The likely explanation is that at level 1.0 many shifted boxes fall outside the image and get clipped or discarded, so the effective corruption is not as severe as at 0.8. The paper does not report the noisy-vs-clean IoU distribution or the fraction of discarded boxes, so the severity ordering cannot be verified. That means the MAE differences in the abstract and the 'severe noise' headline are not a controlled stress test. This is an internal correctness problem, not an external debate.\n\nOther soft spots are smaller but worth noting: COCO is only two levels with no variance reported; alpha and lambda were tuned on subsets of the evaluation data; the few-shot advantage over SSD-Det at 1–3 shots is small and mixed; and the method trades a couple of points at zero/low noise. None of these would be fatal if the monotonicity issue were resolved, but they do mean the paper is currently a conditional accept at best.\n\nWho should read it: anyone working on noisy object detection or using SAM as a preprocessor. It is a serious contribution with a properly described method, and the VOC ablation is a model of transparency. But before I would trust the headline numbers, I would want the authors to plot the actual IoU distribution per noise level, report effective corrupted-box counts, and fix the COCO/hyperparameter issues.\n\nRecommendation: send it to peer review — the idea deserves referee time — but the reviewers should require the noise protocol analysis before publication. I would not cite it in the next year as a result; the claimed robustness gains are currently tethered to an unverified severity scale.","headline":"A genuinely clever, detector-agnostic correction pipeline, but the headline robustness claims rest on a noise protocol whose severity may not be monotonic.","tokens_in":12520,"tokens_out":2565,"would_cite":false,"duration_ms":23709,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"FMG-Det uses SAM and CLIP to fix noisy bounding boxes before training and keeps detectors accurate at high noise levels.","keywords":["object detection","noisy bounding boxes","annotation noise","foundation models","Segment Anything","CLIP","multiple instance learning","few-shot detection"],"falsifier":"Run FMG-Det on a dataset where a large share of noisy boxes have zero overlap with the object, using the stock SAM model; if the detector's mAP at high noise is not better than the same detector trained on the untouched noisy boxes, the claim that SAM/CLIP correction carries the robustness gain is falsified. A cheaper check is to replace SAM's candidate masks with random masks on VOC: the reported gap over SSD-Det should disappear if the masks are the actual cause.","tokens_in":1891,"feed_emoji":"🎯","tokens_out":2498,"duration_ms":88905,"temperature":0.7,"pith_summary":"Object detection models degrade sharply when training bounding boxes are noisy, and more so in few-shot regimes. FMG-Det is a proposed remedy: before training, an offline pipeline prompts SAM with each noisy box and its centerpoint, ranks the resulting candidate masks with CLIP using the ground-truth class text, and keeps the best mask's bounding box unless it shifts too far from the original. The corrected boxes are fed alongside the original noisy boxes into a multiple instance learning detector with a learned interpolation that blends each pair. The paper reports that this raises average performance by 6.8 mAP on COCO and 6.6 mAP on VOC over prior state-of-the-art methods, with the largest gains at high noise levels. A sympathetic reader would care because it offers a simple, detector-agnostic route to training on imperfect human annotations rather than requiring relabeling.","feed_headline":"SAM and CLIP correct noisy boxes, keeping detectors accurate","feed_subtitle":"Offline label fixing plus a learned box blend beats prior robust detectors by 6+ mAP on COCO and VOC.","key_machinery":"The central object is the Foundation Model Correction (FMC) pipeline, a training-free preprocessing step. For each noisy box it produces a diverse set of candidate masks by prompting SAM with both the box and its centerpoint; it scores each candidate by combining SAM's score with a CLIP image-text similarity against the class label; it selects the argmax, converts the winning mask to an axis-aligned box, and rejects corrections whose IoU with the original is below $\\lambda = 0.05$. This pipeline is what carries the robustness gain, since the ablation attributes most of the improvement to it (VOC MAE falls from 36.4 to 17.5). A secondary mechanism is the learned instance-interpolation module, which uses the detector backbone's ROI features to predict a mixing coefficient $\\gamma$ between each noisy and corrected box, letting the training procedure fall back toward the original annotation when the correction is suspect.","core_discovery":"The paper claims that noisy bounding boxes can be corrected well enough by foundation models, without any training, to serve as high-quality training signal for a detector. The correction procedure (FMC) generates candidate masks by calling SAM twice per annotation—once with the noisy box as a prompt and once with its centerpoint—then computes a CLIP score for each masked crop against the class label, combines it with SAM's own mask score via $0.5 c_{j,k} + 0.5 s_{j,k}$, selects the maximum, converts the mask to a box, and discards the correction if its IoU with the noisy box falls below $\\lambda = 0.05$. FMG-Det then feeds both the noisy and corrected boxes to OA-MIL and trains a small network to output $\\gamma = \\sigma(G_\\theta([\\mathrm{ROI}(\\hat{b}_j,x); \\mathrm{ROI}(b_j,x)]))$, producing the interpolated box $b^*_j = \\gamma \\hat{b}_j + (1-\\gamma) b_j$. Under the synthetic noise protocol described in Section 3, the method attains 46.5 mAP on VOC 2007 at noise level 1.0 versus 31.3 for SSD-Det and 19.0 for Faster R-CNN, and 15.7 mAP on COCO at noise 0.8 versus 1.5 for SSD-Det, cutting the paper's mean-absolute-error metric by 6.6 points on VOC and 6.8 on COCO. It also reports the first few-shot robust-detection results, where noise is shown to be more damaging.","pith_inferences":["The paper's own limitation note implies that when a noisy box has no overlap with the object, SAM tends to segment background or a neighbor, and the $\\lambda$ filter then reverts to the noisy annotation; one can infer that on datasets where such zero-overlap errors are common, the expected gain would shrink toward zero or reverse.","Because the synthetic noise is uniform and random, real annotator noise—which tends to be systematic—may be easier for the same pipeline to correct; that would make the reported gains a lower bound, but the paper does not test this.","One could test the interpolation module directly: on corrected boxes that overlap the object, learned $\\gamma$ should be near 1, and on rejected corrections, near 0; the paper does not report these values.","The method's reliance on SAM suggests a domain-shift test: applying the unmodified pipeline to medical or aerial imagery should fail unless SAM is swapped for a domain-tuned variant, matching the paper's stated limitation."],"forward_implications":["At noise level 1.0 on PASCAL VOC, the corrected pipeline reports 46.5 mAP while the prior best baseline reports 31.3, so high-noise annotation sets that were previously unusable become trainable.","On COCO at noise level 0.8, FMG-Det reports 15.7 mAP against 1.5 for SSD-Det, meaning the method keeps detection viable where previous approaches collapsed below 2 mAP.","In few-shot settings, noise is more damaging and FMG-Det still beats the prior best, suggesting the correction pipeline helps most when every training example counts.","Because the correction stage is offline and detector-agnostic, the method can be paired with any two-stage detector and adds minimal training-time computation.","The ablation indicates the FMC pipeline alone already reaches state-of-the-art VOC robustness, with the interpolation module adding a further improvement."],"supporting_citations":[{"why":"Establishes the bounding-box noise protocol and the OA-MIL baseline that FMG-Det builds on.","marker":"[2]"},{"why":"Provides the prior state-of-the-art SSD-Det method that FMG-Det compares against.","marker":"[3]"},{"why":"Supplies SAM, the foundation model that generates candidate masks from box and centerpoint prompts.","marker":"[6]"},{"why":"Supplies CLIP, used to score candidate masks against the ground-truth class text.","marker":"[7]"},{"why":"Defines the few-shot base/novel splits used for the few-shot experiments.","marker":"[20]"},{"why":"Defines Faster R-CNN, the base detector on which all methods in the paper are built.","marker":"[21]"}],"fun_headline_variants":["Foundation models fix noisy boxes, boosting detector robustness","SAM and CLIP correct labels before training, improving few-shot detection","Noisy annotations? FMG-Det uses SAM and CLIP to correct them","Simple pretrained-model fix beats complex robust detectors","Correct boxes with SAM and CLIP, train detectors that are robust"],"cache_read_input_tokens":14336,"weakest_assumption_plain":"The entire robustness gain rests on the assumption that SAM, when prompted with a noisy box and its centerpoint, produces a candidate mask that actually covers the target object; when that fails, the pipeline keeps the original noisy annotation and the main source of improvement is gone.","fun_headline_variants_meta":{"raw":{"variants":["Foundation models fix noisy boxes, boosting detector robustness","SAM and CLIP correct labels before training, improving few-shot detection","Noisy annotations? FMG-Det uses SAM and CLIP to correct them","Simple pretrained-model fix beats complex robust detectors","Correct boxes with SAM and CLIP, train detectors that are robust"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000198,"raw_usage":{"total_tokens":1434,"prompt_tokens":1077,"completion_tokens":357,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":693,"completion_tokens_details":{"reasoning_tokens":271}},"tokens_in":693,"tokens_out":357,"duration_ms":3821,"temperature":1.0,"reasoning_tokens":271,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T12:40:20.493491+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run FMG-Det on a dataset where a large share of noisy boxes have zero overlap with the object, using the stock SAM model; if the detector's mAP at high noise is not better than the same detector trained on the untouched noisy boxes, the claim that SAM/CLIP correction carries the robustness gain is falsified. A cheaper check is to replace SAM's candidate masks with random masks on VOC: the reported gap over SSD-Det should disappear if the masks are the actual cause.","supporting_citations":[{"cited_title":"Zhang et al","cited_arxiv_id":null,"evidence_quote":"Establishes the bounding-box noise protocol and the OA-MIL baseline that FMG-Det builds on."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the prior state-of-the-art SSD-Det method that FMG-Det compares against."},{"cited_title":"We presented a new robust object detection algorithm, FMG-Det, that leverages powerful foundation models to correct bound- ing boxes, mitigating the impact of noisy annotations","cited_arxiv_id":null,"evidence_quote":"Supplies SAM, the foundation model that generates candidate masks from box and centerpoint prompts."},{"cited_title":"Learning trans- ferable visual models from natural language supervi- sion,","cited_arxiv_id":null,"evidence_quote":"Supplies CLIP, used to score candidate masks against the ground-truth class text."},{"cited_title":"A novel benchmark for refinement of noisy localization labels in autolabeled datasets for object detection,","cited_arxiv_id":null,"evidence_quote":"Defines the few-shot base/novel splits used for the few-shot experiments."},{"cited_title":"Dynamic loss decay based robust oriented object detec- tion on remote sensing images with noisy labels,","cited_arxiv_id":null,"evidence_quote":"Defines Faster R-CNN, the base detector on which all methods in the paper are built."}],"review_version":1}