REVIEW 3 major objections 5 minor 1 cited by
IoU Loss for 2D/3D Object Detection
T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read Rotated-box IoU loss lifts 3D detection at strict thresholds on KITTI.
desk verdict Useful integration of rotated IoU loss, but the headline gain is confounded by a threshold change that may cut either way; needs one ablation before the claim holds. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the IoU loss layer, a differentiable module that treats a predicted box and a ground-truth box as polygons or cuboids and returns $1 - \text{IoU}$ as the loss. Its forward pass finds the intersection polygon of two rotated rectangles, sorts the vertices, and computes the polygon area by triangulation; the backward pass supplies gradients through those geometric operations, which standard deep-learning frameworks do not provide. The paper also generalizes the GIoU definition to rotated boxes so that non-overlapping predictions receive a useful gradient. This layer is what lets a detector's training objective match its evaluation metric.
What would settle it
Train each detector under four conditions: L1 at threshold 0.6, L1 at threshold 0.5, LIoU at threshold 0.6, and LIoU at threshold 0.5, then compare AP70 and AP80 on the same validation split. If the threshold-matched pairs perform similarly, the loss itself is inert; if IoU wins at both thresholds, the central claim survives.
Extended reading notes
Core claim
The central claim is that IoU, previously used as a loss only for axis-aligned 2D boxes, can be made differentiable for rotated 2D boxes and for 3D boxes with one yaw angle, and then used directly as the regression loss. The authors' implementation computes the overlap polygon of two rotated rectangles by gathering edge-intersection points and contained corners, sorting them anticlockwise, triangulating, and summing; for cuboids the 2D overlap area is multiplied by the overlap along the height axis. Training with this loss alone, or with the generalized form that also penalizes disjoint boxes, improves the car AP of SECOND, PointPillars, and PointRCNN on the KITTI benchmark compared to the same detectors trained with L1. The improvements grow as the evaluation IoU threshold rises, which the authors read as evidence that the IoU loss is directly optimizing what the benchmark measures.
Load-bearing premise
The reported improvements are assumed to come from replacing the loss, even though the positive-anchor threshold was also lowered from 0.6 to 0.5 in the IoU-loss runs; no baseline trained at 0.5 with L1 or IoU-loss run at 0.6 is provided.
Editorial extensions
If this is right
- Any regression-based 2D or 3D detector can swap its L1 regression head for the proposed loss layer without changing the network architecture.
- Because the loss is scale invariant and couples location, size, and orientation, it should give the largest gains on precisely localized boxes, that is, under high IoU thresholds such as 0.80.
- The same layer covers axis-aligned, rotated 2D, and 3D boxes with one yaw, so it can be reused across bird's-eye-view and point-cloud detection.
- For two-stage detectors like PointRCNN, replacing the second-stage box refinement loss with IoU loss improves refinement quality, so the benefit is not limited to one-stage architectures.
Reading between the lines
- The paper's experimental protocol changes the positive-anchor threshold from 0.6 to 0.5 in the IoU-loss runs; until a threshold-matched comparison is run, part of the gain could come from the extra positive anchors rather than the loss itself.
- The backward pass through a zero-overlap polygon is likely to have zero or poorly defined gradients, so plain $L_{\mathrm{IoU}}$ may be uninformative for badly initialized boxes; the rotated GIoU variant or a smooth surrogate may be the safer default in real training runs.
- If the mechanism is genuinely metric alignment, the same loss layer should transfer to other rotated-box benchmarks such as aerial-image detection, where rotated boxes are standard, and to full 3D rotation once the overlap computation is extended to arbitrary orientations.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes replacing standard L1 box-regression losses with IoU-based losses (LIoU = 1 - IoU and LGIoU) for rotated 2D and yaw-only 3D bounding boxes. It describes IoU computation for rotated rectangles and 3D boxes, implements a custom loss layer with forward and backward passes, and integrates it into SECOND, PointPillars, and PointRCNN. Across KITTI validation and test splits, the authors report consistent AP improvements over L1 baselines, especially at strict IoU thresholds.
Significance. If the empirical claims hold, this is practically significant: it supplies a unified, framework-independent regression loss that aligns the training objective with the evaluation metric for rotated and 3D detection, a case not covered by axis-aligned IoU losses. The paper reports large relative gains at strict thresholds (e.g., up to 34-35% relative improvement at IoU=0.8 for PointRCNN), which would be of broad interest. The loss itself contains no free parameters, and the evaluation uses an independent external benchmark; these are strengths. However, verification is currently blocked by the experimental confound and the lack of released code, so the significance is conditional on the additional experiments requested below.
major comments (3)
- [Sections 5.1 and 5.2, Tables 1-5] The central empirical claim is confounded by a simultaneous change in the positive-anchor threshold. The paper states in Section 5.1: "The only difference is that we decrease the threshold of an anchor be considered as a positive sample during the training from 0.6 to 0.5..." and further notes that the baseline uses 0.6 because "it gives better results than 0.5." Section 5.2 applies the same decrease for PointPillars. Every comparison of LIoU/LGIoU with L1 in Tables 1-5 therefore varies two factors at once. Without an L1 baseline trained at 0.5 or a proposed-loss run trained at 0.6, the reported improvements cannot be attributed to the loss replacement. Please add the missing cross-ablations for at least one detector and report all three thresholds (loss and threshold combinations).
- [Section 3.2, Algorithm 1] Algorithm 1, line 3 computes Areaoverlap = (max(x2, x'2) - min(x1, x'1)) * (max(y1, y'1) - min(y2, y'2)). This is the span of the smallest axis-aligned rectangle enclosing both boxes, not the intersection. The correct axis-aligned intersection width is max(0, min(x2, x'2) - max(x1, x'1)), and the height is analogous. As written, the algorithm overestimates the overlap and can produce a positive "overlap" for disjoint boxes, which can even make the IoU negative. This is a technical error in the specification of the proposed loss layer for the axis-aligned case and should be corrected.
- [Section 4.2.2] The backward pass of the custom IoU loss layer is asserted but not documented. The text says only that "we implement the backward operations for all these functions and we will make the source code public in the future." Since the loss layer is the paper's main contribution, the absence of gradient formulas, implementation, or gradient-checking results prevents independent verification of the training dynamics and leaves the possibility of an incorrect or approximated gradient. The authors should provide the exact backward rules and numerical gradient checks, or make the code available.
minor comments (5)
- [Abstract and Eq. (2)] There are typos: "discrepency" should be "discrepancy", and "Aera" in Eq. (2) should be "Area".
- [Figure 2] The caption says the intersection area is highlighted in gray, while the text for the rotated case says the overlap is highlighted in blue; please align the description.
- [Section 3.4, Eq. (3)] The sentence introducing Eq. (3) mentions both "hoverlap" and "hunion", but the formula uses only "hoverlap". Please define the height intersection explicitly and state whether the height union is needed.
- [Section 5.3 and Tables 5-6] The PointRCNN baseline numbers differ between Table 5 (e.g., AP70 Easy 88.14) and Table 6 (88.88), and the text explains that the authors could not reproduce the published numbers. Please state explicitly which table uses the reproduced model and which uses the official reported model, so readers are not misled.
- [Section 5.4, Table 7] The test-set comparison reports only SECOND+LIoU and not SECOND+LGIoU or the other detectors; please explain why the test submission was limited to this configuration.
Circularity Check
No circularity: LIoU/LGIoU are defined directly from geometric IoU, and the claimed improvements are evaluated on the external KITTI benchmark; the simultaneous threshold change from 0.6 to 0.5 is an experimental confound, not a circular step.
full rationale
The paper's derivation chain is self-contained and non-circular. The IoU computation (Eqs. 1-3, Algorithms 1-2) is a direct geometric construction: intersection area/volume divided by union area/volume, with no free parameters fitted to KITTI. The losses are defined as LIoU = 1 - IoU (Eq. 4) and GIoU = IoU - (Area_C - U)/Area_C (Eq. 5), both monotone transformations of the same geometric IoU; this is an intentional design matching train and test objectives, not a hidden reduction of a prediction to an input. The paper's central empirical claim—that replacing L1 regression with these losses improves AP on KITTI—is evaluated against an external benchmark with published baselines; the improvements are not forced by construction, since training with this loss could fail to improve AP for many reasons (differentiability, optimization, thresholding). No load-bearing step relies on a self-citation: references [1] and [2] are external prior works, and the paper's own contribution is the rotated/3D implementation. The only substantive weakness is the experimental confound that the positive-anchor threshold is lowered from 0.6 to 0.5 in all proposed-loss runs (Sections 5.1 and 5.2), without an L1 baseline at 0.5 or a proposed-loss run at 0.6. That is a correctness/validity concern, not a circular derivation: the reported improvements are not equal to the loss function by definition. Accordingly, no circularity is present.
Assumptions & free parameters
free parameters (1)
- Positive anchor IoU threshold for training with proposed loss =
0.5 (baseline uses 0.6)
assumptions (4)
- standard math Convex polygon intersection and area computation for rotated rectangles is implemented correctly.
- domain assumption The custom backward pass gives correct gradients for the IoU loss layer.
- domain assumption Changes other than the loss, apart from the anchor threshold, do not affect the comparison.
- domain assumption The single-yaw representation for 3D boxes in autonomous driving is appropriate.
Cite this review
Pith. "Pith review of IoU Loss for 2D/3D Object Detection." pith.science (2026). https://pith.science/paper/E7A4ZILR
@misc{pith2026190803851,
author = {Pith},
title = {Pith review of: IoU Loss for 2D/3D Object Detection},
year = {2026},
howpublished = {\url{https://pith.science/paper/E7A4ZILR}},
note = {Machine review of arXiv:1908.03851}
}
abstract
In 2D/3D object detection task, Intersection-over-Union (IoU) has been widely employed as an evaluation metric to evaluate the performance of different detectors in the testing stage. However, during the training stage, the common distance loss (\eg, $L_1$ or $L_2$) is often adopted as the loss function to minimize the discrepancy between the predicted and ground truth Bounding Box (Bbox). To eliminate the performance gap between training and testing, the IoU loss has been introduced for 2D object detection in \cite{yu2016unitbox} and \cite{rezatofighi2019generalized}. Unfortunately, all these approaches only work for axis-aligned 2D Bboxes, which cannot be applied for more general object detection task with rotated Bboxes. To resolve this issue, we investigate the IoU computation for two rotated Bboxes first and then implement a unified framework, IoU loss layer for both 2D and 3D object detection tasks. By integrating the implemented IoU loss into several state-of-the-art 3D object detectors, consistent improvements have been achieved for both bird-eye-view 2D detection and point cloud 3D detection on the public KITTI benchmark.
Figures
Forward citations
Cited by 1 Pith paper
-
Inter-Class Relational Loss for Small Object Detection: A Case Study on License Plates
A new relational loss adds a penalty when a plate's predicted box misses its car, reportedly boosting mAP on two detectors.
Reference graph
Works this paper leans on
-
[1]
Unitbox: An advanced object detection net- work
Jiahui Yu, Yuning Jiang, Zhangyang Wang, Zhimin Cao, and Thomas Huang. Unitbox: An advanced object detection net- work. In Proceedings of the 24th ACM international confer- ence on Multimedia , pages 516–520. ACM, 2016. 1, 2, 3, 4
work page 2016
-
[2]
Generalized in- tersection over union: A metric and a loss for bounding box regression
Hamid Rezatofighi, Nathan Tsoi, JunYoung Gwak, Amir Sadeghian, Ian Reid, and Silvio Savarese. Generalized in- tersection over union: A metric and a loss for bounding box regression. arXiv preprint arXiv:1902.09630, 2019. 1, 2, 3, 4, 5
arXiv 1902
-
[3]
Are we ready for autonomous driving? the kitti vision benchmark suite
Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are we ready for autonomous driving? the kitti vision benchmark suite. In 2012 IEEE Conference on Computer Vision and Pattern Recognition, pages 3354–3361. IEEE, 2012. 1, 3, 4, 5, 8
work page 2012
-
[4]
Fast r-cnn
Ross Girshick. Fast r-cnn. In Proceedings of the IEEE inter- national conference on computer vision , pages 1440–1448,
-
[5]
Faster r-cnn: Towards real-time object detection with region proposal networks
Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. In Advances in neural information pro- cessing systems, pages 91–99, 2015. 1, 2, 3
work page 2015
-
[6]
Ssd: Single shot multibox detector
Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, and Alexander C Berg. Ssd: Single shot multibox detector. In European con- ference on computer vision, pages 21–37. Springer, 2016. 1, 2
work page 2016
-
[7]
You only look once: Unified, real-time object de- tection
Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. You only look once: Unified, real-time object de- tection. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 779–788, 2016. 1, 2, 3, 4
work page 2016
-
[8]
Frustum pointnets for 3d object detection from rgb- d data
Charles R Qi, Wei Liu, Chenxia Wu, Hao Su, and Leonidas J Guibas. Frustum pointnets for 3d object detection from rgb- d data. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 918–927, 2018. 1, 2, 8
work page 2018
Show all 28 references
-
[9]
V oxelnet: End-to-end learning for point cloud based 3d object detection
Yin Zhou and Oncel Tuzel. V oxelnet: End-to-end learning for point cloud based 3d object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4490–4499, 2018. 1, 2, 5, 8
2018
-
[10]
Second: Sparsely embed- ded convolutional detection
Yan Yan, Yuxing Mao, and Bo Li. Second: Sparsely embed- ded convolutional detection. Sensors, 18(10):3337, 2018. 1, 2, 5, 6, 8
2018
-
[11]
Pointpillars: Fast en- coders for object detection from point clouds
Alex H Lang, Sourabh V ora, Holger Caesar, Lubing Zhou, Jiong Yang, and Oscar Beijbom. Pointpillars: Fast en- coders for object detection from point clouds. arXiv preprint arXiv:1812.05784, 2018. 1, 2, 5, 7, 8
2018 arXiv
-
[12]
Pointr- cnn: 3d object proposal generation and detection from point cloud
Shaoshuai Shi, Xiaogang Wang, and Hongsheng Li. Pointr- cnn: 3d object proposal generation and detection from point cloud. In CVPR, 2019. 1, 2, 5, 7, 8
2019
-
[13]
Rich feature hierarchies for accurate object detection and semantic segmentation
Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. Rich feature hierarchies for accurate object detection and semantic segmentation. In Proceedings of the IEEE con- ference on computer vision and pattern recognition , pages 580–587, 2014. 2
2014
-
[14]
Mask r-cnn
Kaiming He, Georgia Gkioxari, Piotr Doll ´ar, and Ross Gir- shick. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision, pages 2961–2969, 2017. 2, 3, 4
2017
-
[15]
Scalable object detection using deep neural networks
Dumitru Erhan, Christian Szegedy, Alexander Toshev, and Dragomir Anguelov. Scalable object detection using deep neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2147–2154,
-
[16]
Dssd: Deconvolutional single shot detector
Cheng-Yang Fu, Wei Liu, Ananth Ranga, Ambrish Tyagi, and Alexander C Berg. Dssd: Deconvolutional single shot detector. arXiv preprint arXiv:1701.06659, 2017. 2
2017 arXiv
-
[17]
Monocular 3d object de- tection for autonomous driving
Xiaozhi Chen, Kaustav Kundu, Ziyu Zhang, Huimin Ma, Sanja Fidler, and Raquel Urtasun. Monocular 3d object de- tection for autonomous driving. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 2147–2156, 2016. 2
2016
-
[18]
Squeezeseg: Convolutional neural nets with recurrent crf for real-time road-object segmentation from 3d lidar point cloud
Bichen Wu, Alvin Wan, Xiangyu Yue, and Kurt Keutzer. Squeezeseg: Convolutional neural nets with recurrent crf for real-time road-object segmentation from 3d lidar point cloud. In 2018 IEEE International Conference on Robotics and Au- tomation (ICRA), pages 1887–1893. IEEE, 2018. 2
2018
-
[19]
Pointnet: Deep learning on point sets for 3d classification and segmentation
Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 652–660,
-
[20]
Improving ob- ject localization with fitness nms and bounded iou loss
Lachlan Tychsen-Smith and Lars Petersson. Improving ob- ject localization with fitness nms and bounded iou loss. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6877–6885, 2018. 3, 6
2018
-
[21]
The pascal visual object classes (voc) challenge
Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. The pascal visual object classes (voc) challenge. International journal of computer vision, 88(2):303–338, 2010. 3
2010
-
[22]
Microsoft coco: Common objects in context
Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision , pages 740–755. Springer, 2014. 3
2014
-
[23]
nuscenes: A mul- timodal dataset for autonomous driving
Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh V ora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Giancarlo Baldan, and Oscar Beijbom. nuscenes: A mul- timodal dataset for autonomous driving. arXiv preprint arXiv:1903.11027, 2019. 4
1903 arXiv
-
[24]
Multi-view 3d object detection network for autonomous driving
Xiaozhi Chen, Huimin Ma, Ji Wan, Bo Li, and Tian Xia. Multi-view 3d object detection network for autonomous driving. In Proceedings of the IEEE Conference on Com- puter Vision and Pattern Recognition , pages 1907–1915,
1907
-
[25]
Joint 3d proposal generation and object detection from view aggregation
Jason Ku, Melissa Mozifian, Jungwook Lee, Ali Harakeh, and Steven L Waslander. Joint 3d proposal generation and object detection from view aggregation. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1–8. IEEE, 2018. 8
2018
-
[26]
Deep continuous fusion for multi-sensor 3d object detection
Ming Liang, Bin Yang, Shenlong Wang, and Raquel Urtasun. Deep continuous fusion for multi-sensor 3d object detection. In Proceedings of the European Conference on Computer Vi- sion (ECCV), pages 641–656, 2018. 8
2018
-
[27]
Ipod: Intensive point-based object detector for point cloud
Zetong Yang, Yanan Sun, Shu Liu, Xiaoyong Shen, and Ji- aya Jia. Ipod: Intensive point-based object detector for point cloud. arXiv preprint arXiv:1812.05276, 2018. 8
2018 arXiv
-
[28]
Frustum convnet: Sliding frustums to aggregate local point-wise features for amodal 3d object detection
Zhixin Wang and Kui Jia. Frustum convnet: Sliding frustums to aggregate local point-wise features for amodal 3d object detection. arXiv preprint arXiv:1903.01864, 2019. 8
1903 arXiv
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.