Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

IoU Loss for 2D/3D Object Detection

T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Pith's one-line read Rotated-box IoU loss lifts 3D detection at strict thresholds on KITTI.

desk verdict Useful integration of rotated IoU loss, but the headline gain is confounded by a threshold change that may cut either way; needs one ablation before the claim holds. read the letter →

arxiv 1908.03851 v1 pith:E7A4ZILR submitted 2019-08-11 cs.CV

classification cs.CV
keywords 3DobjectdetectionIoUlossrotatedboundingboxpointcloudKITTIbenchmarkgeneralizedbird's-eye-viewregression
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to close the gap between how 3D detectors are trained and how they are evaluated. Detectors usually regress box coordinates, sizes, and yaw with L1 or L2 losses, while test-time quality is measured by intersection-over-union, so a low loss does not guarantee a high IoU. The authors show that IoU between two rotated rectangles can be computed as the area of their intersection polygon, extend this to 3D cuboids by multiplying by height overlap, and use the result as a loss. They implement forward and backward passes for a framework-independent layer, define $L_{\mathrm{IoU}} = 1 - \text{IoU}$ and a rotated version of GIoU, and plug it into three LiDAR detectors. The consistent result is higher KITTI car-detection average precision than the L1 baselines, with the largest margins at the strictest IoU thresholds.

What carries the argument

The load-bearing object is the IoU loss layer, a differentiable module that treats a predicted box and a ground-truth box as polygons or cuboids and returns $1 - \text{IoU}$ as the loss. Its forward pass finds the intersection polygon of two rotated rectangles, sorts the vertices, and computes the polygon area by triangulation; the backward pass supplies gradients through those geometric operations, which standard deep-learning frameworks do not provide. The paper also generalizes the GIoU definition to rotated boxes so that non-overlapping predictions receive a useful gradient. This layer is what lets a detector's training objective match its evaluation metric.

What would settle it

Train each detector under four conditions: L1 at threshold 0.6, L1 at threshold 0.5, LIoU at threshold 0.6, and LIoU at threshold 0.5, then compare AP70 and AP80 on the same validation split. If the threshold-matched pairs perform similarly, the loss itself is inert; if IoU wins at both thresholds, the central claim survives.

Watch

Extended reading notes

Core claim

The central claim is that IoU, previously used as a loss only for axis-aligned 2D boxes, can be made differentiable for rotated 2D boxes and for 3D boxes with one yaw angle, and then used directly as the regression loss. The authors' implementation computes the overlap polygon of two rotated rectangles by gathering edge-intersection points and contained corners, sorting them anticlockwise, triangulating, and summing; for cuboids the 2D overlap area is multiplied by the overlap along the height axis. Training with this loss alone, or with the generalized form that also penalizes disjoint boxes, improves the car AP of SECOND, PointPillars, and PointRCNN on the KITTI benchmark compared to the same detectors trained with L1. The improvements grow as the evaluation IoU threshold rises, which the authors read as evidence that the IoU loss is directly optimizing what the benchmark measures.

Load-bearing premise

The reported improvements are assumed to come from replacing the loss, even though the positive-anchor threshold was also lowered from 0.6 to 0.5 in the IoU-loss runs; no baseline trained at 0.5 with L1 or IoU-loss run at 0.6 is provided.

Editorial extensions

If this is right

  • Any regression-based 2D or 3D detector can swap its L1 regression head for the proposed loss layer without changing the network architecture.
  • Because the loss is scale invariant and couples location, size, and orientation, it should give the largest gains on precisely localized boxes, that is, under high IoU thresholds such as 0.80.
  • The same layer covers axis-aligned, rotated 2D, and 3D boxes with one yaw, so it can be reused across bird's-eye-view and point-cloud detection.
  • For two-stage detectors like PointRCNN, replacing the second-stage box refinement loss with IoU loss improves refinement quality, so the benefit is not limited to one-stage architectures.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's experimental protocol changes the positive-anchor threshold from 0.6 to 0.5 in the IoU-loss runs; until a threshold-matched comparison is run, part of the gain could come from the extra positive anchors rather than the loss itself.
  • The backward pass through a zero-overlap polygon is likely to have zero or poorly defined gradients, so plain $L_{\mathrm{IoU}}$ may be uninformative for badly initialized boxes; the rotated GIoU variant or a smooth surrogate may be the safer default in real training runs.
  • If the mechanism is genuinely metric alignment, the same loss layer should transfer to other rotated-box benchmarks such as aerial-image detection, where rotated boxes are standard, and to full 3D rotation once the overlap computation is extended to arbitrary orientations.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes replacing standard L1 box-regression losses with IoU-based losses (LIoU = 1 - IoU and LGIoU) for rotated 2D and yaw-only 3D bounding boxes. It describes IoU computation for rotated rectangles and 3D boxes, implements a custom loss layer with forward and backward passes, and integrates it into SECOND, PointPillars, and PointRCNN. Across KITTI validation and test splits, the authors report consistent AP improvements over L1 baselines, especially at strict IoU thresholds.

Significance. If the empirical claims hold, this is practically significant: it supplies a unified, framework-independent regression loss that aligns the training objective with the evaluation metric for rotated and 3D detection, a case not covered by axis-aligned IoU losses. The paper reports large relative gains at strict thresholds (e.g., up to 34-35% relative improvement at IoU=0.8 for PointRCNN), which would be of broad interest. The loss itself contains no free parameters, and the evaluation uses an independent external benchmark; these are strengths. However, verification is currently blocked by the experimental confound and the lack of released code, so the significance is conditional on the additional experiments requested below.

major comments (3)
  1. [Sections 5.1 and 5.2, Tables 1-5] The central empirical claim is confounded by a simultaneous change in the positive-anchor threshold. The paper states in Section 5.1: "The only difference is that we decrease the threshold of an anchor be considered as a positive sample during the training from 0.6 to 0.5..." and further notes that the baseline uses 0.6 because "it gives better results than 0.5." Section 5.2 applies the same decrease for PointPillars. Every comparison of LIoU/LGIoU with L1 in Tables 1-5 therefore varies two factors at once. Without an L1 baseline trained at 0.5 or a proposed-loss run trained at 0.6, the reported improvements cannot be attributed to the loss replacement. Please add the missing cross-ablations for at least one detector and report all three thresholds (loss and threshold combinations).
  2. [Section 3.2, Algorithm 1] Algorithm 1, line 3 computes Areaoverlap = (max(x2, x'2) - min(x1, x'1)) * (max(y1, y'1) - min(y2, y'2)). This is the span of the smallest axis-aligned rectangle enclosing both boxes, not the intersection. The correct axis-aligned intersection width is max(0, min(x2, x'2) - max(x1, x'1)), and the height is analogous. As written, the algorithm overestimates the overlap and can produce a positive "overlap" for disjoint boxes, which can even make the IoU negative. This is a technical error in the specification of the proposed loss layer for the axis-aligned case and should be corrected.
  3. [Section 4.2.2] The backward pass of the custom IoU loss layer is asserted but not documented. The text says only that "we implement the backward operations for all these functions and we will make the source code public in the future." Since the loss layer is the paper's main contribution, the absence of gradient formulas, implementation, or gradient-checking results prevents independent verification of the training dynamics and leaves the possibility of an incorrect or approximated gradient. The authors should provide the exact backward rules and numerical gradient checks, or make the code available.
minor comments (5)
  1. [Abstract and Eq. (2)] There are typos: "discrepency" should be "discrepancy", and "Aera" in Eq. (2) should be "Area".
  2. [Figure 2] The caption says the intersection area is highlighted in gray, while the text for the rotated case says the overlap is highlighted in blue; please align the description.
  3. [Section 3.4, Eq. (3)] The sentence introducing Eq. (3) mentions both "hoverlap" and "hunion", but the formula uses only "hoverlap". Please define the height intersection explicitly and state whether the height union is needed.
  4. [Section 5.3 and Tables 5-6] The PointRCNN baseline numbers differ between Table 5 (e.g., AP70 Easy 88.14) and Table 6 (88.88), and the text explains that the authors could not reproduce the published numbers. Please state explicitly which table uses the reproduced model and which uses the official reported model, so readers are not misled.
  5. [Section 5.4, Table 7] The test-set comparison reports only SECOND+LIoU and not SECOND+LGIoU or the other detectors; please explain why the test submission was limited to this configuration.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: LIoU/LGIoU are defined directly from geometric IoU, and the claimed improvements are evaluated on the external KITTI benchmark; the simultaneous threshold change from 0.6 to 0.5 is an experimental confound, not a circular step.

full rationale

The paper's derivation chain is self-contained and non-circular. The IoU computation (Eqs. 1-3, Algorithms 1-2) is a direct geometric construction: intersection area/volume divided by union area/volume, with no free parameters fitted to KITTI. The losses are defined as LIoU = 1 - IoU (Eq. 4) and GIoU = IoU - (Area_C - U)/Area_C (Eq. 5), both monotone transformations of the same geometric IoU; this is an intentional design matching train and test objectives, not a hidden reduction of a prediction to an input. The paper's central empirical claim—that replacing L1 regression with these losses improves AP on KITTI—is evaluated against an external benchmark with published baselines; the improvements are not forced by construction, since training with this loss could fail to improve AP for many reasons (differentiability, optimization, thresholding). No load-bearing step relies on a self-citation: references [1] and [2] are external prior works, and the paper's own contribution is the rotated/3D implementation. The only substantive weakness is the experimental confound that the positive-anchor threshold is lowered from 0.6 to 0.5 in all proposed-loss runs (Sections 5.1 and 5.2), without an L1 baseline at 0.5 or a proposed-loss run at 0.6. That is a correctness/validity concern, not a circular derivation: the reported improvements are not equal to the loss function by definition. Accordingly, no circularity is present.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central claim rests on correct geometric intersection, correct gradients, and fair experimental isolation. No free parameters are fitted inside the loss itself, but one training hyperparameter was changed between conditions. No new physical or conceptual entities are introduced.

free parameters (1)
  • Positive anchor IoU threshold for training with proposed loss = 0.5 (baseline uses 0.6)
    Changed simultaneously with the loss replacement in all experiments; no ablation controls for it, so it is a hand-set experimental variable that confounds attribution of the reported gains.
assumptions (4)
  • standard math Convex polygon intersection and area computation for rotated rectangles is implemented correctly.
    Algorithm 2 step 3 assumes a correct convex-polygon vertex extraction and sorting method, but the vertex determination and area summation are not detailed or proven.
  • domain assumption The custom backward pass gives correct gradients for the IoU loss layer.
    Section 4.2.2 states that backward operations were implemented, but no derivative formulas, chain-rule decomposition, or numerical gradient checks are provided. Training behavior depends on gradient correctness.
  • domain assumption Changes other than the loss, apart from the anchor threshold, do not affect the comparison.
    Sections 5.1 and 5.2 claim the same config files and training strategies as the baselines, but for PointRCNN the authors state they could not reproduce the published numbers, so the baseline itself is an unverified local reproduction.
  • domain assumption The single-yaw representation for 3D boxes in autonomous driving is appropriate.
    Section 3.3 assumes objects lie on a flat road plane, reducing rotation to one degree of freedom, which is standard for KITTI but domain-specific and would not generalize to full 3D rotation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of IoU Loss for 2D/3D Object Detection." pith.science (2026). https://pith.science/paper/E7A4ZILR

@misc{pith2026190803851,
  author       = {Pith},
  title        = {Pith review of: IoU Loss for 2D/3D Object Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/E7A4ZILR}},
  note         = {Machine review of arXiv:1908.03851}
}
abstract

In 2D/3D object detection task, Intersection-over-Union (IoU) has been widely employed as an evaluation metric to evaluate the performance of different detectors in the testing stage. However, during the training stage, the common distance loss (\eg, $L_1$ or $L_2$) is often adopted as the loss function to minimize the discrepancy between the predicted and ground truth Bounding Box (Bbox). To eliminate the performance gap between training and testing, the IoU loss has been introduced for 2D object detection in \cite{yu2016unitbox} and \cite{rezatofighi2019generalized}. Unfortunately, all these approaches only work for axis-aligned 2D Bboxes, which cannot be applied for more general object detection task with rotated Bboxes. To resolve this issue, we investigate the IoU computation for two rotated Bboxes first and then implement a unified framework, IoU loss layer for both 2D and 3D object detection tasks. By integrating the implemented IoU loss into several state-of-the-art 3D object detectors, consistent improvements have been achieved for both bird-eye-view 2D detection and point cloud 3D detection on the public KITTI benchmark.

Figures

Figures reproduced from arXiv: 1908.03851 by the authors.

Figure 1
Figure 1. An example of 3D car detection results from different [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. IoU computation for 2D: axis-aligned and rotated bound [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. An example of 3D car detection results with different methods, where the left is from original SECOND method and the right is [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Inter-Class Relational Loss for Small Object Detection: A Case Study on License Plates

    cs.CV 2025-08 unverdicted novelty 5.0 of 10

    A new relational loss adds a penalty when a plate's predicted box misses its car, reportedly boosting mAP on two detectors.

Reference graph

Works this paper leans on

28 extracted references · 17 canonical work pages · cited by 1 Pith paper

  1. [1]

    Unitbox: An advanced object detection net- work

    Jiahui Yu, Yuning Jiang, Zhangyang Wang, Zhimin Cao, and Thomas Huang. Unitbox: An advanced object detection net- work. In Proceedings of the 24th ACM international confer- ence on Multimedia , pages 516–520. ACM, 2016. 1, 2, 3, 4

  2. [2]

    Generalized in- tersection over union: A metric and a loss for bounding box regression

    Hamid Rezatofighi, Nathan Tsoi, JunYoung Gwak, Amir Sadeghian, Ian Reid, and Silvio Savarese. Generalized in- tersection over union: A metric and a loss for bounding box regression. arXiv preprint arXiv:1902.09630, 2019. 1, 2, 3, 4, 5

  3. [3]

    Are we ready for autonomous driving? the kitti vision benchmark suite

    Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are we ready for autonomous driving? the kitti vision benchmark suite. In 2012 IEEE Conference on Computer Vision and Pattern Recognition, pages 3354–3361. IEEE, 2012. 1, 3, 4, 5, 8

  4. [4]

    Fast r-cnn

    Ross Girshick. Fast r-cnn. In Proceedings of the IEEE inter- national conference on computer vision , pages 1440–1448,

  5. [5]

    Faster r-cnn: Towards real-time object detection with region proposal networks

    Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. In Advances in neural information pro- cessing systems, pages 91–99, 2015. 1, 2, 3

  6. [6]

    Ssd: Single shot multibox detector

    Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, and Alexander C Berg. Ssd: Single shot multibox detector. In European con- ference on computer vision, pages 21–37. Springer, 2016. 1, 2

  7. [7]

    You only look once: Unified, real-time object de- tection

    Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. You only look once: Unified, real-time object de- tection. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 779–788, 2016. 1, 2, 3, 4

  8. [8]

    Frustum pointnets for 3d object detection from rgb- d data

    Charles R Qi, Wei Liu, Chenxia Wu, Hao Su, and Leonidas J Guibas. Frustum pointnets for 3d object detection from rgb- d data. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 918–927, 2018. 1, 2, 8

Show all 28 references
  1. [9]

    V oxelnet: End-to-end learning for point cloud based 3d object detection

    Yin Zhou and Oncel Tuzel. V oxelnet: End-to-end learning for point cloud based 3d object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4490–4499, 2018. 1, 2, 5, 8

  2. [10]

    Second: Sparsely embed- ded convolutional detection

    Yan Yan, Yuxing Mao, and Bo Li. Second: Sparsely embed- ded convolutional detection. Sensors, 18(10):3337, 2018. 1, 2, 5, 6, 8

  3. [11]

    Pointpillars: Fast en- coders for object detection from point clouds

    Alex H Lang, Sourabh V ora, Holger Caesar, Lubing Zhou, Jiong Yang, and Oscar Beijbom. Pointpillars: Fast en- coders for object detection from point clouds. arXiv preprint arXiv:1812.05784, 2018. 1, 2, 5, 7, 8

  4. [12]

    Pointr- cnn: 3d object proposal generation and detection from point cloud

    Shaoshuai Shi, Xiaogang Wang, and Hongsheng Li. Pointr- cnn: 3d object proposal generation and detection from point cloud. In CVPR, 2019. 1, 2, 5, 7, 8

  5. [13]

    Rich feature hierarchies for accurate object detection and semantic segmentation

    Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of the IEEE con- ference on computer vision and pattern recognition , pages 580–587, 2014. 2

  6. [14]

    Mask r-cnn

    Kaiming He, Georgia Gkioxari, Piotr Doll ´ar, and Ross Gir- shick. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision, pages 2961–2969, 2017. 2, 3, 4

  7. [15]

    Scalable object detection using deep neural networks

    Dumitru Erhan, Christian Szegedy, Alexander Toshev, and Dragomir Anguelov. Scalable object detection using deep neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2147–2154,

  8. [16]

    Dssd: Deconvolutional single shot detector

    Cheng-Yang Fu, Wei Liu, Ananth Ranga, Ambrish Tyagi, and Alexander C Berg. Dssd: Deconvolutional single shot detector. arXiv preprint arXiv:1701.06659, 2017. 2

  9. [17]

    Monocular 3d object de- tection for autonomous driving

    Xiaozhi Chen, Kaustav Kundu, Ziyu Zhang, Huimin Ma, Sanja Fidler, and Raquel Urtasun. Monocular 3d object de- tection for autonomous driving. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 2147–2156, 2016. 2

  10. [18]

    Squeezeseg: Convolutional neural nets with recurrent crf for real-time road-object segmentation from 3d lidar point cloud

    Bichen Wu, Alvin Wan, Xiangyu Yue, and Kurt Keutzer. Squeezeseg: Convolutional neural nets with recurrent crf for real-time road-object segmentation from 3d lidar point cloud. In 2018 IEEE International Conference on Robotics and Au- tomation (ICRA), pages 1887–1893. IEEE, 2018. 2

  11. [19]

    Pointnet: Deep learning on point sets for 3d classification and segmentation

    Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 652–660,

  12. [20]

    Improving ob- ject localization with fitness nms and bounded iou loss

    Lachlan Tychsen-Smith and Lars Petersson. Improving ob- ject localization with fitness nms and bounded iou loss. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6877–6885, 2018. 3, 6

  13. [21]

    The pascal visual object classes (voc) challenge

    Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. The pascal visual object classes (voc) challenge. International journal of computer vision, 88(2):303–338, 2010. 3

  14. [22]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision , pages 740–755. Springer, 2014. 3

  15. [23]

    nuscenes: A mul- timodal dataset for autonomous driving

    Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh V ora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: A mul- timodal dataset for autonomous driving. arXiv preprint arXiv:1903.11027, 2019. 4

  16. [24]

    Multi-view 3d object detection network for autonomous driving

    Xiaozhi Chen, Huimin Ma, Ji Wan, Bo Li, and Tian Xia. Multi-view 3d object detection network for autonomous driving. In Proceedings of the IEEE Conference on Com- puter Vision and Pattern Recognition , pages 1907–1915,

  17. [25]

    Joint 3d proposal generation and object detection from view aggregation

    Jason Ku, Melissa Mozifian, Jungwook Lee, Ali Harakeh, and Steven L Waslander. Joint 3d proposal generation and object detection from view aggregation. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1–8. IEEE, 2018. 8

  18. [26]

    Deep continuous fusion for multi-sensor 3d object detection

    Ming Liang, Bin Yang, Shenlong Wang, and Raquel Urtasun. Deep continuous fusion for multi-sensor 3d object detection. In Proceedings of the European Conference on Computer Vi- sion (ECCV), pages 641–656, 2018. 8

  19. [27]

    Ipod: Intensive point-based object detector for point cloud

    Zetong Yang, Yanan Sun, Shu Liu, Xiaoyong Shen, and Ji- aya Jia. Ipod: Intensive point-based object detector for point cloud. arXiv preprint arXiv:1812.05276, 2018. 8

  20. [28]

    Frustum convnet: Sliding frustums to aggregate local point-wise features for amodal 3d object detection

    Zhixin Wang and Kui Jia. Frustum convnet: Sliding frustums to aggregate local point-wise features for amodal 3d object detection. arXiv preprint arXiv:1903.01864, 2019. 8

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.