{"id":"cef22661-de9f-4bc9-80ef-e72b3230069d","arxiv_id":"1908.05641","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"IoU-balanced classification and localization losses, which up-weight high-IoU examples and down-weight low-IoU examples during training, improve AP by about 1.0 to 1.7 points on COCO test-dev for several single-stage detectors.","lead":"This paper proposes two loss-function changes for single-stage object detectors, reweighting classification and localization losses by how well each predicted box overlaps its ground truth. The changes improve detection accuracy on COCO, PASCAL VOC, and Cityscapes without changing the network architecture or inference speed.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. 4 normalization makes IoU-balanced classification loss algebraically equal to standard CE; without explicit stop-gradient the loss is a no-op, contradicting reported gains.","rationale":"The paper's central claim rests on both proposed losses. The classification loss formula in Eq. 4 has a normalization that cancels the weighting, making the total loss algebraically identical to standard cross-entropy unless the weights are detached. This is a concrete algebraic issue, not merely a question of optimization stability. The reader's weakest_assumption focused on gradient-detached localization weighting and Bounded IoU approximation, which are secondary concerns; my concern is more fundamental because it concerns the definition of the proposed method itself. Without an explicit stop-gradient for the classification weights, the loss cannot have the effect claimed in the component analysis. The localization loss (Eq. 6 with manually adjusted w_loc) is well-defined, so the paper may be salvageable with a corrected formula or a clarified detachment rule; however, as written, the method is not fully specified and the reported gains cannot be reproduced from the given equations. This warrants moving the verdict from CONDITIONAL to UNVERDICTED until the authors provide the missing specification or correct the formula.","tokens_in":14064,"tokens_out":6697,"duration_ms":66847,"concrete_test":"Re-derive the total positive classification loss from Eq. 3-4 with and without stop-gradient on w_i. If the algebraic reduction to standard CE holds, implement the exact Eq. 3-4 in an autodiff framework on a toy example and verify that the gradient w.r.t. p_i equals the standard CE gradient when weights are not detached. Then check whether the reported Table 2 '+0.7 AP for IoU-Cls' can be reproduced with the intended detached-weight variant; if not, the formula requires correction.","verdict_should_be":"UNVERDICTED","load_bearing_attack":"In Eqs. 3-4, the positive-classification term is Σ_i w_i·CE_i with w_i = iou_i^η · (Σ_j CE_j) / (Σ_j iou_j^η · CE_j). Then the total positive loss reduces algebraically to Σ_j CE_j, exactly the standard CE loss, independent of IoU. If gradients flow through w_i, the loss and its gradients are identical to standard CE, so the IoU-balanced classification loss cannot produce the +0.7 AP attributed to it in Table 2. If the weights are meant to be detached (stop-gradient), this is not stated for the classification loss, unlike the explicit statement for the localization loss in Sec. 3.3: 'We constrains that the gradients are not propagated from wi(ioui) to lm i'. With detached weights, the forward value would still equal standard CE, but gradients would be reweighted; either way, the paper must specify the detachment and/or correct the normalization (e.g., divide by mean iou^η rather than by the CE-weighted sum). As written, the method is not fully defined, and the central claim is unverifiable.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes two loss modifications for single-stage object detectors. The IoU-balanced classification loss (Eqs. 3-4) is intended to reweight positive classification examples by the current predicted IoU, and the IoU-balanced localization loss (Eqs. 5-6) reweights smooth L1 by a power of IoU while stopping gradient flow through the weight. Experiments with SSD, RetinaNet, and FoveaBox on COCO, PASCAL VOC, and Cityscapes report consistent AP gains of about 1.0-1.7 points on COCO test-dev and larger gains at high IoU thresholds, with unchanged inference speed.","tokens_in":14337,"tokens_out":6643,"duration_ms":61562,"significance":"If the mechanism is correct, the contribution is practically valuable because it improves localization without architectural changes or inference cost. The paper's strengths are the breadth of the empirical study (three datasets, multiple detectors and backbones, a two-stage comparison), the generalization check using hyperparameters fixed from COCO, and the explicit gradient analysis for the localization loss. However, the classification loss as written is a mathematical no-op, so the central claim currently rests on an underspecified or incorrect formula.","major_comments":[{"comment":"As written, the IoU-balanced classification loss is algebraically identical to the standard cross-entropy loss on the positive examples. Substituting Eq. (4) into Eq. (3), the positive-term sum Σ_{i∈Pos} w_i CE_i equals [Σ_j CE_j / Σ_j iou_j^η CE_j] · Σ_i iou_i^η CE_i = Σ_j CE_j, so the forward loss does not depend on IoU at all. If gradients are allowed to flow through w_i, the gradients with respect to the classification logits are also those of standard CE. If w_i is intended to be detached (as is explicitly stated for the localization loss in Sec. 3.3), the forward value is still standard CE and only the gradient is reweighted; this is not stated for the classification loss. The paper must specify the exact implementation, including any stop-gradient, and correct the normalization (for example, using w_i = iou_i^η / mean_j(iou_j^η)) so that the loss is not a mathematical no-op. Without this correction, the +0.7 AP attributed to the IoU-balanced classification loss in Table 2 is unexplained.","section":"Sec. 3.2, Eqs. (3)-(4)"},{"comment":"The localization loss uses the model's own current IoU as a per-example weight, with gradients detached from the weight, creating a feedback loop whose dynamics are not analyzed. The gradient formulas in Eqs. (13)-(14) are derived from the Bounded IoU approximation with the additional assumptions δ=0.111 and w_t=w_s, so they describe an upper-bound surrogate rather than the exact IoU used in training. The paper should either provide evidence that the exact-IoU gradient behaves in the same way (for example, a plot of the exact gradient against d for representative boxes) or state clearly that the analysis is only heuristic. This matters because the claimed improvement mechanism is precisely that the weight up-weights inliers and down-weights outliers.","section":"Sec. 3.3, Eqs. (5)-(14)"},{"comment":"All reported results are single training runs without error bars or significance tests. The headline improvements are 0.7-1.7 AP, a range where typical seed-to-seed variation for these detectors is non-negligible, and the hyperparameters η and λ are selected on COCO val (Table 3) before reporting test-dev numbers. I recommend reporting results over at least three seeds for the main comparisons, or otherwise providing evidence that the gains exceed run-to-run noise.","section":"Sec. 4, Tables 1-7"}],"minor_comments":[{"comment":"There are several language slips: 'We constrains' should be 'We constrain', and 'it's variants' should be 'its variants'.","section":"Sec. 1 and Sec. 3.3"},{"comment":"The symbol N is used both for the number of positive examples and as the upper limit of summation; please define N explicitly as the number of positive examples.","section":"Sec. 3.2, Eq. (4)"},{"comment":"Equation (7) redefines w_i after Eq. (6) has already defined it; please clarify that Eq. (7) is an alternative normalization strategy and is not used in the main experiments.","section":"Sec. 3.3, Eqs. (6)-(7)"},{"comment":"The caption and axis labels in Figure 2 are small, and the curves for different λ values are hard to distinguish in print; please enlarge the figure or use distinct line styles.","section":"Figure 2"},{"comment":"Reference [23] contains a typo: 'Beyound' should be 'Beyond'.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The algebraic issue in Sec. 3.2 is serious and must be resolved before publication. The empirical results are promising, and the paper could be publishable after the loss is correctly specified and the experiments are confirmed with the corrected formulation. I would also ask the editor to require the authors to clarify the exact stop-gradient implementation, since the current text is ambiguous."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThis is a mostly straightforward empirical paper: replace the standard cross-entropy and smooth-L1 losses in single-stage detectors with IoU-weighted versions, and pick up 1–2 points of AP. The idea is simple, architecture-free, and the experiments are extensive and consistent across COCO test-dev, PASCAL VOC, Cityscapes, several backbones, and detectors including SSD, RetinaNet, and FoveaBox. The qualitative analysis showing that the classification-score distribution shifts toward high-IoU detections is exactly the effect you'd want. That part is worth taking seriously.\n\nThe problem is the definition of the classification loss. In Eqs. 3–4, the per-example weight is normalized so that the weighted sum of positive cross-entropy terms is algebraically equal to the unnormalized sum. If gradients flow through the weights, the loss and its gradient are exactly standard CE—a no-op. The paper never says the weights are detached for classification. It explicitly says for the localization loss that gradients are not propagated from the weight to the regression output, but no such statement appears for the classification loss. As written, the method is either a no-op or an unspecified reweighting trick. The reported +0.7 AP from the classification loss can only arise if the weights are detached, so the paper needs one sentence of clarification and ideally a derivation of the actual gradient used.\n\nOther soft spots are minor by comparison: single runs without variance, hyperparameters tuned on COCO val and fixed elsewhere (a reasonable generalization check), no code release despite the abstract promising it, and no direct comparison against GHM or Libra R-CNN in the main tables. The localization-loss gradient analysis uses the Bounded IoU approximation from Fitness NMS, which is acceptable but should be framed as an upper-bound analysis, not exact.\n\nMy take: the central empirical claim is plausible, and the specification fix is trivial. This is a legitimately useful contribution for practitioners, but it needs a careful revision before it can be fully trusted. A serious referee should see it; desk rejection would be a mistake.","headline":"IoU-weighted losses are a plausible, simple +1-2 AP gain for single-stage detectors, but the classification loss as written is an algebraic no-op unless the weights are detached—needs a clear revision.","tokens_in":14822,"tokens_out":4700,"would_cite":false,"duration_ms":43539,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that reweighting the classification and localization losses of single-stage detectors with the IoU of each predicted box improves localization accuracy substantially—AP by 1.0–1.7% and AP75 by 1.0–2.4% on COCO…","keywords":["IoU-balanced classification loss","IoU-balanced localization loss","object detection","accurate localization","class imbalance","example mining","smooth L1 loss","single-stage detectors"],"falsifier":"Train a single-stage detector with the same two loss weights but allow gradients to flow through the IoU weight, or freeze the weights at their first-iteration values; if the AP gains disappear or training diverges, the detachment and the moving weight are load-bearing. Alternatively, log per-example gradient norms during training: the paper's claim implies high-IoU examples should contribute more gradient and low-IoU examples less than with plain smooth L1.","tokens_in":13901,"feed_emoji":"🎯","tokens_out":5542,"duration_ms":43818,"temperature":0.7,"pith_summary":"This paper argues that the two standard loss terms in single-stage object detectors—cross-entropy for classification and smooth L1 for bounding-box regression—are themselves a source of localization error, and that fixing the loss functions alone can recover much of the accuracy that more complex architectures are built to gain. To that end it proposes IoU-balanced losses: a classification loss that up-weights positive examples whose predicted boxes have high intersection-over-union with ground truth, and a localization loss that does the same for the regression gradients while suppressing the gradients of low-IoU outliers. On COCO test-dev the losses raise AP by 1.0–1.7 points and AP75 by 1.0–2.4 points across SSD, RetinaNet, and FoveaBox, with no change to architecture or inference time. The paper also shows the gains transfer to PASCAL VOC and Cityscapes, with the largest gains at high IoU thresholds (AP80/AP90), exactly where localization quality matters most.","feed_headline":"Reweight losses by IoU to lift AP75 by up to 2.4%","feed_subtitle":"Training SSD, RetinaNet, and FoveaBox with IoU-weighted losses improves localization accuracy at zero inference cost.","key_machinery":"The central mechanism is a per-example weight that depends on the IoU between the currently predicted box and its ground-truth box, applied to both loss terms. For classification, the weight is iou^η normalized so the total positive-example loss stays constant; for localization, the weight is w_loc·iou^λ applied to each smooth L1 term, with the gradient detached so the weight acts as a fixed training signal rather than a differentiable function of the prediction. The paper derives closed-form gradient upper bounds using the Bounded IoU approximation, which treats IoU as a simplified function of coordinate offsets, and uses them to show why the scheme raises inlier gradients and suppresses outlier gradients.","core_discovery":"The central claim is that localization accuracy in single-stage detectors is limited less by the network design than by the loss functions that train it. Standard cross-entropy treats every positive anchor equally, so the classifier learns high scores even for poorly localized boxes; standard smooth L1 lets outliers dominate the regression gradient. The paper's IoU-balanced classification loss multiplies the cross-entropy term of each positive example by a normalized weight proportional to the regressed IoU raised to a power η, and the IoU-balanced localization loss multiplies the smooth L1 terms by a weight w_loc·IoU^λ that up-weights inliers and down-weights outliers. The gradient is not propagated through the IoU weight. With the best settings (η=1.5, λ=1.5) the combined losses improve AP by 1.0–1.7% and AP75 by 1.0–2.4% on COCO test-dev across SSD, RetinaNet, and FoveaBox, and improve AP80/90 by 1.6–3.9% on PASCAL VOC, with no architectural change and no inference cost.","pith_inferences":["Because the reweighting is a drop-in training-time change, the same recipe could apply to other dense prediction tasks where confidence and spatial accuracy are jointly trained, such as keypoint detection or instance segmentation, provided an IoU-like quality measure can be computed per positive example.","The gradient-detachment choice is the part most worth probing: allowing gradients to flow through the IoU weight would turn the loss into a feedback loop, and the paper gives no stability analysis. A controlled experiment varying only the detach operation would isolate whether detachment is load-bearing.","The use of Bounded IoU rather than exact IoU in the gradient derivation suggests the exact gradient may behave differently near the δ boundary; measuring the true gradient norms during training would show whether the claimed inlier/outlier rebalancing holds in practice.","Since the gains are largest at high IoU thresholds, the losses may be a cheaper alternative to cascade or refinement modules when the goal is strict localization, for example in scene text or aerial imagery."],"forward_implications":["On COCO test-dev, applying IoU-balanced losses to SSD300/512, RetinaNet, and FoveaBox improves AP by 1.0–1.7% and AP75 by 1.0–2.4%.","The improvements concentrate at high IoU thresholds: AP80 and AP90 rise by 1.6–2.6% on COCO and 1.6–3.9% on PASCAL VOC, showing the losses mainly buy tighter localization.","The losses generalize across datasets (COCO, PASCAL VOC, Cityscapes) and across backbones (ResNet-18 to ResNeXt-101) with consistent AP gains of about 1–1.5%.","The correlation between classification scores and localization improves: the paper reports higher average classification scores for high-IoU detections and lower scores for low-IoU detections after training with the IoU-balanced classification loss.","Two-stage detectors benefit less (about 0.4–0.6% AP), consistent with the claim that the mismatch is smaller when proposals are already well localized."],"supporting_citations":[{"why":"SSD is one of the single-stage detectors whose AP and AP75 improve when the proposed losses are applied.","marker":"[1]"},{"why":"RetinaNet is the main testbed for ablations and backbone studies, with focal loss as the classification baseline.","marker":"[3]"},{"why":"FoveaBox is the anchor-free detector used to show the losses generalize beyond anchor-based methods.","marker":"[23]"},{"why":"Supplies the Bounded IoU approximation that the paper uses to derive closed-form gradient upper bounds for the IoU-balanced localization loss.","marker":"[18]"},{"why":"Documents that outlier gradients dominate the localization subnetwork, the problem the IoU-balanced localization loss is designed to counter.","marker":"[6]"},{"why":"Introduces smooth L1 loss, the base localization loss that the proposed loss modifies.","marker":"[12]"},{"why":"MS COCO is the primary evaluation dataset whose AP and AP75 metrics measure the reported improvements.","marker":"[24]"},{"why":"Provides the baseline reimplementations and training protocol used for fair comparison in all experiments.","marker":"[27]"}],"fun_headline_variants":["IoU-balanced losses sharpen single-stage object detection","Reweight detection losses by IoU for better localization","IoU-aware training lifts AP75 by up to 2.4% on COCO","Fix detector loss functions to improve localization accuracy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The approach relies on the assumption that weighting each example by how well the model currently localizes it—while preventing the gradient from adjusting that weight—will continue improving the model instead of destabilizing training.","fun_headline_variants_meta":{"raw":{"variants":["IoU-balanced losses sharpen single-stage object detection","Reweight detection losses by IoU for better localization","IoU-aware training lifts AP75 by up to 2.4% on COCO","Fix detector loss functions to improve localization accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000264,"raw_usage":{"total_tokens":1690,"prompt_tokens":1121,"completion_tokens":569,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":737,"completion_tokens_details":{"reasoning_tokens":501}},"tokens_in":737,"tokens_out":569,"duration_ms":5450,"temperature":1.0,"reasoning_tokens":501,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T13:06:58.221148+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a single-stage detector with the same two loss weights but allow gradients to flow through the IoU weight, or freeze the weights at their first-iteration values; if the AP gains disappear or training diverges, the detachment and the moving weight are load-bearing. Alternatively, log per-example gradient norms during training: the paper's claim implies high-IoU examples should contribute more gradient and low-IoU examples less than with plain smooth L1.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"SSD is one of the single-stage detectors whose AP and AP75 improve when the proposed losses are applied."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"RetinaNet is the main testbed for ablations and backbone studies, with focal loss as the classification baseline."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"FoveaBox is the anchor-free detector used to show the losses generalize beyond anchor-based methods."},{"cited_title":"Tychsen-Smith, L","cited_arxiv_id":null,"evidence_quote":"Supplies the Bounded IoU approximation that the paper uses to derive closed-form gradient upper bounds for the IoU-balanced localization loss."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Documents that outlier gradients dominate the localization subnetwork, the problem the IoU-balanced localization loss is designed to counter."},{"cited_title":"Girshick, Fast r-cnn, in: Proceedings of the IEEE international conference on computer vision, 2015, pp","cited_arxiv_id":null,"evidence_quote":"Introduces smooth L1 loss, the base localization loss that the proposed loss modifies."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"MS COCO is the primary evaluation dataset whose AP and AP75 metrics measure the reported improvements."}],"review_version":1}