REVIEW 3 major objections 6 minor 43 references
Fast Point R-CNN
T0 review · 3 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read Fast Point R-CNN fuses voxel grids with raw point coordinates in a two-stage detector, reporting state-of-the-art KITTI 3D and BEV accuracy while running at 15 FPS.
desk verdict Solid two-stage LiDAR detector with careful ablations, but the 'state-of-the-art' claim is contradicted by its own Table 1. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the attention-based fusion module inside RefinerNet. For each point inside a proposal (plus a 0.3 m margin), the convolutional feature map from VoxelRPN is indexed at the point's projected BEV location and concatenated with a 128-dimensional MLP embedding of the point's canonized coordinates; a sigmoid attention computed from the convolutional features gates this combined representation before two pointwise MLP layers and a max-pool aggregate the box into a fixed-size feature. This lets each raw point retain exact position while still receiving context and receptive-field information from the voxel convolutional stream. A second supporting mechanism is the canonized-corner regression target: points and ground-truth corners are rotated and translated into the proposal's coordinate frame, ordering the eight corners deterministically, and the network regresses offsets to those ordered corners rather than a direct box parameterization.
What would settle it
Run the full pipeline on the KITTI validation split with RefinerNet fed (a) VoxelRPN proposals and (b) a high-recall alternative (e.g., ground-truth proposals or a proposal set with deliberately lowered recall via weakened augmentation), and compare final AP0.7 and AP0.8 for both near and distant objects. If feeding higher-recall proposals does not substantially improve final AP, refinement is doing little beyond rescoring; if lowering VoxelRPN recall by a few points collapses final AP, the claimed two-stage benefit is contingent on first-stage recall being near-perfect.
Extended reading notes
Core claim
The central claim is that the speed-accuracy trade-off in 3D detection can be shifted by splitting the problem along representational lines rather than architectural lines. A voxel-based region proposal network (VoxelRPN), built from a small number of 3D convolutions followed by 2D convolutions, produces a compact set of initial boxes quickly and with high recall. The second stage (RefinerNet) then re-examines the raw points inside each proposed box, canonizes them by rotating and translating into the proposal frame, and fuses an MLP embedding of each point's coordinates with the convolutional feature from VoxelRPN at the point's projected location. A sigmoid attention gate applied to the convolutional feature emphasizes informative context before two pointwise MLP layers and a max-pool aggregate the box's points. This fusion restores the precise localization lost during voxelization and strided convolutions; the paper reports that the refinement raises moderate 3D AP0.7 from 76.64 to 79.00 on the KITTI validation split, with large gains at higher IoU thresholds (AP0.8 rising from 42.6 to 52.95) and for distant objects in the 30-50 m range.
Load-bearing premise
The central assumption is that the first-stage VoxelRPN already proposes essentially all true objects (over 95% recall at 0.5 BEV IoU within the top 30 boxes for cars), because RefinerNet can only re-score and re-shape boxes that the first stage finds; any object missed by VoxelRPN is missed by the whole system.
Editorial extensions
If this is right
- If the central claim holds, high-quality 3D detection no longer requires RGB image input or multi-sensor fusion; LiDAR alone can match or exceed the accuracy of sensor-fusion methods while running faster.
- Refining only a small set of proposals keeps the second stage cheap: the full pipeline runs at 0.065 s per frame on a Tesla P40, opening the door to real-time use in autonomous driving.
- The refinement stage provides its largest gains precisely where it matters most: high-IoU metrics (AP0.8) and distant objects (30-50 m), which are the failure modes most relevant to safety.
- The two-stage decoupling means future improvements to first-stage recall (more anchors, higher resolution, better augmentation) translate directly into final-AP gains, since the refinement stage only re-shapes boxes the first stage already finds.
- The method's speed advantage is complementary to sparse-convolution and TensorRT-style engineering, suggesting further speedups are possible without sacrificing accuracy.
Reading between the lines
- The coordinate-plus-context attention fusion could generalize beyond detection to other point-cloud tasks like semantic segmentation or tracking, where the same tension between precise geometry and contextual understanding exists.
- Because RefinerNet only sees points inside the proposed box plus a small margin, the system's recall ceiling is set entirely by VoxelRPN; for pedestrian and cyclist classes, where the paper reports lower first-stage recall, a denser or scale-aware first stage would be the natural next step.
- The reported 15 FPS figure is on a Tesla P40 without sparse convolution or TensorRT; combining these complementary acceleration techniques could push the method toward the speed of the fastest single-stage detectors while keeping the refinement gain.
- The canonized-corner loss may be a reusable recipe for any 3D box regressor that must handle orientation ambiguity, since it removes the corner-ordering uncertainty that plagues direct corner regression.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes Fast Point R-CNN, a two-stage LiDAR-based 3D object detector. The first stage, VoxelRPN, voxelizes the point cloud and uses a lightweight combination of 3D and 2D convolutions to produce a small set of high-recall proposals. The second stage, RefinerNet, crops raw points around each proposal, fuses point coordinates with convolutional features via an attention mechanism, and applies a light PointNet to refine the box corners using a canonized corner regression target. The method is evaluated on the KITTI benchmark for car, pedestrian, and cyclist detection, with ablations on the validation split. On the KITTI test set it reports 75.73 moderate 3D AP for cars at 15 FPS on a Tesla P40, and the authors claim state-of-the-art performance in both 3D and BEV detection.
Significance. If the technical claims hold, the architecture is a useful engineering contribution: it demonstrates that a voxel-based proposal stage plus a raw-point refinement stage can be competitive with much heavier point-wise detectors, and the ablation study gives clear evidence that each component—3D convolutions, higher resolution input, MIXUP augmentation, multi-angle anchors, coordinate features, and the attention fusion module—contributes to the final accuracy. The canonized corner loss and the analysis of distant-object improvement (Table 5) are informative. However, the headline claim of state-of-the-art accuracy is not supported by the paper's own Table 1, and the efficiency claim is presented without the hardware/software context needed for comparison. The absence of code or multiple-run statistics further limits verification. The work is nevertheless a solid, reproducible-in-principle two-stage detector that should be accepted if the claims are calibrated and the stated missing analyses are provided.
major comments (3)
- [Abstract and Section 4.2, Table 1] The assertion that Fast Point R-CNN "achieves state-of-the-arts" is not supported by the numbers in Table 1. On the KITTI primary metric (3D AP@0.7 moderate), the method scores 75.73, which is lower than PointRCNN's 75.76. In BEV moderate it ties PointPillars at 86.10, and on BEV hard it is lower (78.17 vs 79.83). There is no row in Table 1 in which the method is strictly ahead of all listed methods. Please revise the abstract and Section 4.2 to describe the results as competitive with state-of-the-art, or state precisely the metric and difficulty level for which a state-of-the-art claim is made.
- [Section 4.2 and Table 1] The speed claim needs to be framed more carefully. The paper says Fast Point R-CNN runs at 15 FPS, but Table 1 lists PointPillars at 0.016 s (about 62 FPS) and SECOND at 0.05 s (20 FPS). Direct comparison is complicated by different GPUs (Tesla P40 vs GTX 1080Ti) and by PointPillars' use of TensorRT, which the paper acknowledges. As written, however, the abstract's "state-of-the-arts with a 15FPS detection rate" implies a Pareto advantage in both accuracy and speed that Table 1 does not show. Please report the exact hardware and inference optimizations used for each method and avoid implying that the method is both the fastest and the most accurate in the comparison.
- [Section 3.4 and Section 5.3, Table 5] The two-stage design can only refine proposals produced by VoxelRPN, and the paper justifies this by stating that VoxelRPN recall exceeds 95% at 0.5 BEV IoU in the top 30 boxes for the car class. No recall numbers are given for pedestrians, cyclists, or distant objects. Table 5 shows that VoxelRPN's 3D AP0.7 at 30-50 m is only 51.99, and the fact that RefinerNet improves it to 58.41 does not tell us what fraction of distant objects were missed entirely by the first stage and therefore unrecoverable. Please report VoxelRPN recall by class and by distance range, and analyze how many missed proposals are recoverable by RefinerNet; this is necessary to support the general claim that the framework is effective beyond the car class in near-range settings.
minor comments (6)
- [Section 4.1, Implementation Details] The car anchor size is reported as ha=1.73, wa=0.6, la=0.8 meters, which is identical to the pedestrian anchor size and physically implausible for KITTI cars; please confirm the correct anchor dimensions and correct the text if this is a typo.
- [Section 4.1, Implementation Details] For pedestrian and cyclist, the text lists anchor sizes ha=1.73, wa=0.6, la=0.8 and ha=1.73, wa=0.6, la=1.76; please clarify whether the car anchor is actually the same as the pedestrian anchor and explain how these anchor sizes were chosen.
- [Figure 2] The notation "3×" and "5×" near the 2D convolution blocks is not explained in the caption or the text; please specify the number of repeated layers and the meaning of the stride tuples such as /(2,2,2).
- [Section 5.2, Table 4] The fusion baseline labeled "Concatenation" is not fully specified: it is unclear whether the coordinate feature and convolutional feature are concatenated before or after the PointNet MLPs and where the attention mask is applied in the proposed module. Please provide a precise description so the ablation is reproducible.
- [Section 5.2, RoI Align comparison] The rotated RoI align baseline is described as using 4096D MLP layers after pooling with size 8×4; please state whether the same training schedule, positive-proposal threshold, and data augmentation were used as in RefinerNet, so the comparison is apples-to-apples.
- [General] No code or pretrained models are provided, and the performance difference from PointRCNN on the primary metric is 0.03 AP; releasing code or at least detailed inference scripts would materially improve reproducibility and help readers verify the reported results.
Circularity Check
No circularity: the paper reports benchmark results on KITTI and does not rename fitted parameters as predictions.
full rationale
Fast Point R-CNN's central claims are accuracy and speed measurements obtained by training on KITTI and evaluating on the KITTI test server. The loss functions (Eqs. 1-6) define training objectives; they do not by construction determine the reported AP values, which are external benchmark measurements. Ablation studies (Tables 3-6) compare independently trained model variants, and no parameter fitted to a subset of the evaluation data is then presented as a prediction of closely related quantities. The only notable flaw is that the abstract's 'state-of-the-arts' claim is not supported by the paper's own Table 1 (PointRCNN achieves a higher 3D moderate AP), but that is a correctness or framing issue, not a circularity. Self-citations such as IPOD are not load-bearing for the method's derivation, and no uniqueness theorem or hidden ansatz is imported from prior work. Therefore the score is 0.
Assumptions & free parameters
free parameters (6)
- Car anchor size (h, w, l) =
1.73, 0.6, 0.8 meters
- Voxel size =
0.1 x 0.1 x 0.2 m
- Points per voxel =
6
- Class imbalance weight gamma =
10
- Smooth L1 sigma =
3
- MIXUP objects per frame =
20
assumptions (3)
- domain assumption The KITTI dataset annotations and evaluation server are reliable indicators of real-world detection quality.
- domain assumption Random rotations and translations of ground-truth boxes during augmentation preserve the physical plausibility of the scene.
- domain assumption The deep networks used converge to a good local optimum with the given hyperparameters.
Cite this review
Pith. "Pith review of Fast Point R-CNN." pith.science (2026). https://pith.science/paper/UR7MFELN
@misc{pith2026190802990,
author = {Pith},
title = {Pith review of: Fast Point R-CNN},
year = {2026},
howpublished = {\url{https://pith.science/paper/UR7MFELN}},
note = {Machine review of arXiv:1908.02990}
}
read the original abstract
We present a unified, efficient and effective framework for point-cloud based 3D object detection. Our two-stage approach utilizes both voxel representation and raw point cloud data to exploit respective advantages. The first stage network, with voxel representation as input, only consists of light convolutional operations, producing a small number of high-quality initial predictions. Coordinate and indexed convolutional feature of each point in initial prediction are effectively fused with the attention mechanism, preserving both accurate localization and context information. The second stage works on interior points with their fused feature for further refining the prediction. Our method is evaluated on KITTI dataset, in terms of both 3D and Bird's Eye View (BEV) detection, and achieves state-of-the-arts with a 15FPS detection rate.
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
-
[1]
YOLO3D: End-to-end real-time 3D Oriented Object Bounding Box Detection from LiDAR Point Cloud
Waleed Ali, Sherif Abdelkarim, Mohamed Zahran, Mah- moud Zidan, and Ahmad El Sallab. Yolo3d: End-to-end real-time 3d oriented object bounding box detection from li- dar point cloud. arXiv:1808.02350, 2018
work page Pith review arXiv 2018
-
[2]
BirdNet: a 3D Object Detection Framework from LiDAR information
Jorge Beltran, Carlos Guindel, Francisco Miguel Moreno, Daniel Cruzado, Fernando Garcia, and Arturo de la Escalera. Birdnet: a 3d object detection framework from lidar informa- tion. arXiv:1805.01195, 2018
work page Pith review arXiv 2018
-
[3]
Monocular 3d object de- tection for autonomous driving
Xiaozhi Chen, Kaustav Kundu, Ziyu Zhang, Huimin Ma, Sanja Fidler, and Raquel Urtasun. Monocular 3d object de- tection for autonomous driving. In CVPR, 2016
work page 2016
-
[4]
3d object proposals for accurate object class detection
Xiaozhi Chen, Kaustav Kundu, Yukun Zhu, Andrew G Berneshawi, Huimin Ma, Sanja Fidler, and Raquel Urtasun. 3d object proposals for accurate object class detection. In NIPS, 2015
work page 2015
-
[5]
Multi-view 3d object detection network for autonomous driving
Xiaozhi Chen, Huimin Ma, Ji Wan, Bo Li, and Tian Xia. Multi-view 3d object detection network for autonomous driving. In CVPR, 2017
2017
-
[6]
Fully-convolutional point networks for large-scale point clouds
Rethage Dario, Wald Johanna, Sturm Jrgen, Navab Nassir, and Tombari Federico. Fully-convolutional point networks for large-scale point clouds. In ECCV, 2018
work page 2018
-
[7]
Cut, paste and learn: Surprisingly easy synthesis for instance de- tection
Debidatta Dwibedi, Ishan Misra, and Martial Hebert. Cut, paste and learn: Surprisingly easy synthesis for instance de- tection. In ICCV, 2017
2017
-
[8]
Are we ready for autonomous driving? the kitti vision benchmark suite
Andreas Geiger, Philip Lenz, and Raquel Urtasun. Are we ready for autonomous driving? the kitti vision benchmark suite. In CVPR, 2012
2012
Show all 43 references
-
[9]
Fast r-cnn
Ross Girshick. Fast r-cnn. In ICCV, 2015
2015
-
[10]
3d semantic segmentation with submanifold sparse convolutional networks
Benjamin Graham, Martin Engelcke, and Laurens van der Maaten. 3d semantic segmentation with submanifold sparse convolutional networks. CVPR, 2018
2018
-
[11]
Mask r-cnn
Kaiming He, Georgia Gkioxari, Piotr Doll ´ar, and Ross Gir- shick. Mask r-cnn. In ICCV, 2017
2017
-
[12]
Adam: A method for stochastic optimization
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv:1412.6980, 2014
2014 arXiv
-
[13]
Roarnet: A robust 3d object detection based on region approximation refinement
Youngwook Paul Kwon Kiwoo Shin and Masayoshi Tomizuka. Roarnet: A robust 3d object detection based on region approximation refinement. arXiv:1811.03818, 2018
2018 arXiv
-
[14]
Hypernet: Towards accurate region proposal generation and joint object detection
Tao Kong, Anbang Yao, Yurong Chen, and Fuchun Sun. Hypernet: Towards accurate region proposal generation and joint object detection. In CVPR, 2016
2016
-
[15]
Joint 3d proposal generation and ob- ject detection from view aggregation
Jason Ku, Melissa Mozifian, Jungwook Lee, Ali Harakeh, and Steven Waslander. Joint 3d proposal generation and ob- ject detection from view aggregation. IROS, 2018
2018
-
[16]
Pointpillars: Fast encoders for object detection from point clouds
Alex H Lang, Sourabh V ora, Holger Caesar, Lubing Zhou, Jiong Yang, and Oscar Beijbom. Pointpillars: Fast encoders for object detection from point clouds. arXiv:1812.05784, 2018
2018 arXiv
-
[17]
3d fully convolutional network for vehicle detection in point cloud
Bo Li. 3d fully convolutional network for vehicle detection in point cloud. In IROS, 2017
2017
-
[18]
Vehicle detection from 3d lidar using fully convolutional network
Bo Li, Tianlei Zhang, and Tian Xia. Vehicle detection from 3d lidar using fully convolutional network. Robotics: Sci- ence and Systems, 2016
2016
-
[19]
Pointcnn
Yangyan Li, Rui Bu, Mingchao Sun, and Baoquan Chen. Pointcnn. NIPS, 2018
2018
-
[20]
Deep continuous fusion for multi-sensor 3d object detection
Ming Liang, Bin Yang, Shenlong Wang, and Raquel Urtasun. Deep continuous fusion for multi-sensor 3d object detection. In ECCV, 2018
2018
-
[21]
Feature pyramid networks for object detection
Tsung-Yi Lin, Piotr Doll ´ar, Ross B Girshick, Kaiming He, Bharath Hariharan, and Serge J Belongie. Feature pyramid networks for object detection. In CVPR, 2017
2017
-
[22]
Ssd: Single shot multibox detector
Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, and Alexander C Berg. Ssd: Single shot multibox detector. In ECCV, 2016
2016
-
[23]
Fast and furi- ous: Real time end-to-end 3d detection, tracking and motion forecasting with a single convolutional net
Wenjie Luo, Bin Yang, and Raquel Urtasun. Fast and furi- ous: Real time end-to-end 3d detection, tracking and motion forecasting with a single convolutional net. In CVPR, 2018
2018
-
[24]
V oxnet: A 3d con- volutional neural network for real-time object recognition
Daniel Maturana and Sebastian Scherer. V oxnet: A 3d con- volutional neural network for real-time object recognition. In IROS, 2015
2015
-
[25]
Qi, Wei Liu, Chenxia Wu, Hao Su, and Leonidas J
Charles R. Qi, Wei Liu, Chenxia Wu, Hao Su, and Leonidas J. Guibas. Frustum pointnets for 3d object detec- tion from rgb-d data. In CVPR, 2018
2018
-
[26]
Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas
Charles R. Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. CVPR, 2017
2017
-
[27]
Qi, Hao Su, Matthias Nießner, Angela Dai, Mengyuan Yan, and Leonidas J Guibas
Charles R. Qi, Hao Su, Matthias Nießner, Angela Dai, Mengyuan Yan, and Leonidas J Guibas. V olumetric and multi-view cnns for object classification on 3d data. In CVPR, 2016
2016
-
[28]
Qi, Li Yi, Hao Su, and Leonidas J Guibas
Charles R. Qi, Li Yi, Hao Su, and Leonidas J Guibas. Point- net++: Deep hierarchical feature learning on point sets in a metric space. In NIPS, 2017
2017
-
[29]
Pointr- cnn: 3d object proposal generation and detection from point cloud
Shaoshuai Shi, Xiaogang Wang, and Hongsheng Li. Pointr- cnn: 3d object proposal generation and detection from point cloud. arXiv:1812.04244, 2018
2018 arXiv
-
[30]
Training region-based object detectors with online hard ex- ample mining
Abhinav Shrivastava, Abhinav Gupta, and Ross Girshick. Training region-based object detectors with online hard ex- ample mining. In CVPR, 2016
2016
-
[31]
Complex-yolo: Real-time 3d object detec- tion on point clouds
Martin Simon, Stefan Milz, Karl Amende, and Horst- Michael Gross. Complex-yolo: Real-time 3d object detec- tion on point clouds. arXiv:1803.06199, 2018
2018 arXiv
-
[32]
Sliding shapes for 3d ob- ject detection in depth images
Shuran Song and Jianxiong Xiao. Sliding shapes for 3d ob- ject detection in depth images. In ECCV, 2014
2014
-
[33]
Deep sliding shapes for amodal 3d object detection in rgb-d images
Shuran Song and Jianxiong Xiao. Deep sliding shapes for amodal 3d object detection in rgb-d images. In CVPR, 2016
2016
-
[34]
Multi-view convolutional neural networks for 3d shape recognition
Hang Su, Subhransu Maji, Evangelos Kalogerakis, and Erik Learned-Miller. Multi-view convolutional neural networks for 3d shape recognition. In ICCV, 2015
2015
-
[35]
Multi-view 3d models from single images with a convolu- tional network
Maxim Tatarchenko, Alexey Dosovitskiy, and Thomas Brox. Multi-view 3d models from single images with a convolu- tional network. In ECCV, 2016
2016
-
[36]
Dynamic graph cnn for learning on point clouds
Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E Sarma, Michael M Bronstein, and Justin M Solomon. Dynamic graph cnn for learning on point clouds. arXiv:1801.07829, 2018
2018 arXiv
-
[37]
3d shapenets: A deep representation for volumetric shapes
Zhirong Wu, Shuran Song, Aditya Khosla, Fisher Yu, Lin- guang Zhang, Xiaoou Tang, and Jianxiong Xiao. 3d shapenets: A deep representation for volumetric shapes. In CVPR, 2015
2015
-
[38]
Pointfu- sion: Deep sensor fusion for 3d bounding box estimation
Danfei Xu, Dragomir Anguelov, and Ashesh Jain. Pointfu- sion: Deep sensor fusion for 3d bounding box estimation. In CVPR, 2018
2018
-
[39]
Second: Sparsely embed- ded convolutional detection
Yan Yan, Yuxing Mao, and Bo Li. Second: Sparsely embed- ded convolutional detection. Sensors, 2018
2018
-
[40]
Pixor: Real- time 3d object detection from point clouds
Bin Yang, Wenjie Luo, and Raquel Urtasun. Pixor: Real- time 3d object detection from point clouds. In CVPR, 2018
2018
-
[41]
Ipod: Intensive point-based object detector for point cloud
Zetong Yang, Yanan Sun, Shu Liu, Xiaoyong Shen, and Ji- aya Jia. Ipod: Intensive point-based object detector for point cloud. arXiv:1812.05276, 2018
2018 arXiv
-
[42]
mixup: Beyond empirical risk minimiza- tion
Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimiza- tion. In ICLR, 2017
2017
-
[43]
V oxelnet: End-to-end learning for point cloud based 3d object detection
Yin Zhou and Oncel Tuzel. V oxelnet: End-to-end learning for point cloud based 3d object detection. In CVPR, 2018
2018
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.