Pith. sign in

REVIEW 4 major objections 6 minor 16 references

LiFT: Lightweight, FPGA-tailored 3D object detection based on LiDAR data

T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read LiFT is a 3D LiDAR detector designed entirely around FPGA constraints, showing that real-time embedded accuracy and a tight compute budget can coexist.

desk verdict Solid detection result with clean ablations, but the real-time FPGA claim is an unverified design target, not a measured outcome. read the letter →

arxiv 2501.11159 v1 pith:TFV4QCQP submitted 2025-01-19 cs.CV cs.AReess.IV

Konrad Lis , Tomasz Kryjak , Marek Gorgon This is my paper · ORCID
classification cs.CVcs.AReess.IV
keywords 3DobjectdetectionLiDARFPGAINT8quantizationsparseconvolutionPillarFeatureNetNuScenesembeddedvision
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

LiFT is a 3D object detector for LiDAR point clouds, designed from the start to run on a low-power FPGA rather than a GPU. The paper's claim is that the constraints such a platform imposes—roughly 30 GMAC per point cloud, INT8 quantization of all weights and activations, processing on a 2D pillar grid instead of 3D voxels, and almost no skip connections—can be satisfied without sacrificing competitive accuracy. With a computational cost of 20.73 GMAC (billion multiply-accumulate operations), LiFT reports 51.84% mAP (mean average precision) and 61.01% NDS (nuScenes detection score) on the NuScenes validation set, ranking first among detectors that meet the same budget. If this stands, it closes a gap in embedded autonomy: real-time LiDAR perception on an FPGA-class device becomes a realistic target rather than a GPU-only capability.

What carries the argument

The argument is carried by four interacting mechanisms. DBPFN is a modified Pillar Feature Net that concatenates max-pooled and min-pooled features, with the ReLU removed so min pooling sees meaningful negative values; this preserves more information about the distribution of points inside each pillar at essentially zero added compute. The input quantization scheme splits each of the XYZ coordinates into a coarse 8-bit part and a detail residual, keeping localization resolution far finer than the 40 cm a naive 8-bit encoding of a 108 m range would give. Reparameterizable sparse convolutions train with short skip connections and then fold them into a single convolution at inference, so the deployed graph has almost no skip connections. The whole network, including the detection head, is sparse and operates on 2D pillars, which keeps both the compute count and the on-chip buffering needed for Im2Col small.

What would settle it

Implement LiFT on the target FPGA device and measure end-to-end latency per point cloud: if it cannot sustain 10 point clouds per second under INT8 sparse convolutions, the central real-time claim is falsified. Independently, re-running the reported training and ablation pipeline and finding that DBPFN adds less than the reported accuracy would refute the method's novelty.

Watch

Extended reading notes

Core claim

The central discovery is that a fully sparse, 2D-cell detector built around three novel mechanisms—the Dual-Bound Pillar Feature Net (DBPFN), coarse-plus-detail INT8 coordinate encoding, and reparameterizable sparse convolutions—can outperform the previous best embedded-budget detectors while staying under 30 GMAC. In the paper's own comparison, LiFT beats the next-best comparable detector by 1.51 percentage points in mAP (51.84% versus 50.33%) and by 0.17 points in NDS, at a cost only 0.7 GMAC higher. The paper also reports that replacing DBPFN with the standard Pillar Feature Net costs 1.55 mAP and 1.14 NDS, making the new encoder the largest single contributor to the gain.

Load-bearing premise

The load-bearing premise is that a 30 GMAC per-point-cloud budget, computed from a specific FPGA deep-learning processor's peak rate, really translates into real-time operation once memory traffic and control overhead are included; the paper measures no hardware.

Editorial extensions

If this is right

  • A 3D detector can meet the real-time budget of a single mid-range FPGA deep-learning processor (roughly 30 GMAC per point cloud) and still exceed previous embedded-complexity detectors on the NuScenes benchmark.
  • The 30 GMAC ceiling becomes a concrete design target for embedded LiDAR perception, and the reported 9.27 GMAC margin suggests other researchers can trade complexity for accuracy.
  • Fully sparse 2D-cell architectures, with reparameterizable convolutions and a sparse head, are a viable alternative to dense or 3D-voxel designs under strict compute limits.
  • The ablation ordering implies that the pillar encoder DBPFN is the highest-value component, while multiscale fusion is essential: removing it costs 9.43 mAP, so any future FPGA port must preserve both.
  • If the results reproduce, a fully quantized INT8 detector can keep localization resolution at the centimeter scale by splitting coordinates into coarse and detail parts, a technique that costs nothing at inference.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The real-time claim is not yet tested on hardware; a logical next experiment is to implement LiFT on the referenced FPGA deep-learning processor and measure throughput, latency, and energy, since the 20.73 GMAC count excludes data movement and Im2Col overhead.
  • The coarse/detail INT8 coordinate split is a general idea that could transfer to other point-cloud tasks such as segmentation or occupancy prediction, wherever uniform 8-bit quantization of coordinates loses spatial resolution; the paper only evaluates it for 3D detection.
  • Because the paper compares against detectors obeying the same budget but notes LiFT still trails unconstrained state-of-the-art by a wide margin, a natural follow-up is sensor fusion with camera or radar to close that gap while keeping the FPGA constraint.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes LiFT, a LiDAR-based 3D object detector designed under explicit FPGA-oriented constraints: INT8 quantization, 2D pillar processing, limited skip connections, and a 30 GMAC budget derived from the AMD/Xilinx DPU B4096. LiFT uses a fully sparse 2D convolutional backbone and head, a Dual-Bound Pillar Feature Net (DBPFN) that adds min pooling to the standard PFN, reparameterizable sparse convolutions, and a coarse/detail input coordinate quantization scheme. On the NuScenes validation set, the authors report 51.84% mAP and 61.01% NDS at 20.73 GMAC, and they claim it is the best detector meeting their constraints. An FPGA implementation is stated as future work in Section 5.

Significance. If the accuracy and complexity numbers are reproducible, LiFT is a valuable reference for lightweight LiDAR detection: it demonstrates that a fully sparse, INT8-quantized 2D detector can exceed 50 mAP on NuScenes at under 30 GMAC, and it provides an ablation study quantifying the contributions of the proposed components. The hardware-induced constraints derived in Section 3.1 are a useful framework for hardware-aware algorithm design. However, the absence of any FPGA implementation means the "real-time, FPGA-tailored" claim is a design goal rather than a demonstrated result.

major comments (4)
  1. [§3.1, §3.5, §5] The paper's central claim that LiFT is "FPGA-tailored" for real-time operation is not supported by experimental evidence. Section 3.1 derives a 30 GMAC budget from the dense AMD/Xilinx DPU B4096 (2048 MAC/cycle at 300 MHz), but Section 3.5 states that LiFT "consists entirely of sparse convolutions, including the head." The DPU is a dense CNN accelerator; no argument or measurement shows that it can execute sparse submanifold convolutions at the assumed efficiency. Sparse convolution also incurs non-MAC overheads (active-site gathering, coordinate bookkeeping, DRAM traffic) that are absent from the 20.73 GMAC figure. Section 5 explicitly says FPGA implementation is future work. Consequently, the "real-time" and "FPGA-tailored" claims are unverified. Please either provide an FPGA implementation with latency/throughput measurements or substantially revise the title, abstract, and conclusions to present LiFT as a hardware-constrained algorithmic design rather than a validated FPGA solution.
  2. [§4 (experiments), §3.5] The manuscript claims LiFT is "fully quantized" with INT8 weights and activations, but the experimental section gives no details about the quantization pipeline: no quantization-aware training scheme, no calibration procedure, no definitions of quantization ranges, and no comparison of floating-point versus quantized accuracy. It is therefore unclear whether the reported mAP/NDS come from the actual INT8 model or from a floating-point model with simulated quantization. This is load-bearing for the central accuracy claim, because the stated contribution is a design that achieves high accuracy under INT8 constraints. Please add the quantization details and report both floating-point and quantized metrics.
  3. [Table 1, §4.1] Table 1 compares LiFT with SPADE and SPADE+ using GMAC and accuracy numbers taken from those papers, but the GMAC counting convention is not defined. If the PFE (which in LiFT includes DBPFN) or data preprocessing are excluded or included differently across methods, the reported margin and the "ranks first" claim may not be meaningful. Please specify the exact set of operations included in the GMAC count (e.g., PFE, sparse convolutions, head, upsampling) and ideally re-compute GMACs for all compared detectors with a single script.
  4. [Table 2, §4.2] The DBPFN ablation compares DBPFN against the original PFN, but the two differ in two respects: min pooling is added and ReLU is removed. The paper attributes the mAP gain to the "dual-bound" mechanism, but the improvement could be due entirely to ReLU removal or to the interaction between the two changes. Please add an ablation that isolates min pooling with ReLU retained, or ReLU removal with only max pooling, so that the contribution of the proposed dual-bound idea is not confounded.
minor comments (6)
  1. [§3.5] The sentence "The detector operates on 2D cells and consists entirely of sparse convolutions, including the head" is not strictly accurate because the DBPFN encoder is a per-point MLP rather than a sparse convolution; please rephrase to clarify that the backbone and head are sparse convolutions.
  2. [§4.1] No error bars or multiple random seeds are reported. Given the small margins in Table 1 (e.g., 0.17% NDS), the statistical significance of the ranking is unclear; please report variance or explicitly note the single-run nature of the experiments.
  3. [§3.4] The claim that the proposed quantization gives "effective localization resolution below 2 mm" is based on the quantization step of the detail feature, but this does not bound the detector's localization error, which is also limited by the 15 cm pillar size and the center head's offset regression; please rephrase to say "input coordinate quantization resolution" to avoid overclaiming.
  4. [Abstract] The phrase "sub-2mm localization resolution" in the abstract has the same issue as the Section 3.4 claim; please adjust it to be consistent with the suggested rephrasing.
  5. [References] Reference [1] is an AMD/Xilinx web page; please provide a more complete citation including the version and access date.
  6. [Throughout] There are several typographical issues (e.g., Section 2: "the later most common" should be "the latter"; Section 3.1: "1GM AC= 109M AC" should be "10^9") and spacing artifacts from PDF extraction; please ensure the camera-ready version is clean.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: LiFT's accuracy is held-out benchmark evidence and its GMAC budget is an external, specification-derived constraint; the real-time FPGA claim is unverified but not circular.

full rationale

The paper's derivation chain is: (i) derive algorithm constraints from DPU throughput, on-chip memory limits, and an external Vitis AI model-zoo observation; (ii) design LiFT within those constraints; (iii) measure mAP/NDS on the held-out NuScenes-val split and count GMAC; (iv) compare against other detectors meeting the same sub-30-GMAC constraint. Each link is externally anchored. The accuracy figures are benchmark results, not outputs of a fitted model, and the ablations (Table 2) isolate the contribution of each component against meaningful baselines. The sub-2mm localization statement is an arithmetic consequence of the coarse/detail coordinate split and the 8-bit quantization of the detail residual, stated as a design property rather than as a predicted result; it is not a load-bearing 'prediction' that is secretly equivalent to its input. No fitted parameter is renamed as a prediction, and no load-bearing self-citation appears: reference [13] is background related work only, and the 30 GMAC budget is justified from AMD/Xilinx DPU B4096 specifications and an external reference [1], not from the authors' own prior results. The absence of an actual FPGA implementation means the real-time throughput claim is unverified, but that is an evidence gap or correctness risk, not circularity.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

LiFT is an engineering contribution. Its central claims rest on hand-selected hyperparameters (pillar size, channel widths, layer counts) and standard FPGA-domain assumptions about DPU throughput, sparse convolution efficiency, and INT8 accuracy. No new physical entities are introduced.

free parameters (6)
  • Pillar size = 15 cm x 15 cm
    Hand-chosen cell size for the 2D pillar grid; balances resolution and computational cost.
  • PFE output feature count = 64
    Hand-chosen width for the Dual-Bound Pillar Feature Net output.
  • Stage channel widths = 64, 64, 128, 128
    Hand-chosen channel counts for the four backbone stages.
  • RepSubMConv layers per stage = 6, 12, 6, 6
    Hand-chosen depths for each stage of the sparse backbone.
  • Input quantization bit width = 8 bits (INT8)
    Chosen to match FPGA DPU support; the coordinate split uses a factor of 2^-8 per dimension.
  • Point cloud range = [-54, 54] m x [-54, 54] m x [-5, 3] m
    Standard NuScenes range used by other detectors; sets the coarse quantization step.
assumptions (5)
  • domain assumption The Kria K26 platform with a single DPU B4096 at 300 MHz sustains 614.4 GMAC/s, so a 10 pcd/s real-time target implies a per-point-cloud budget near 61.44 GMAC, conservatively set to 30 GMAC.
    Section 3.1: this hardware-derived budget drives the entire design; the paper calls it an approximation.
  • domain assumption Sparse convolutions cut computation by 50-80% versus dense convolutions in 3D detectors.
    Section 3.2: used to justify a fully sparse backbone and head.
  • domain assumption RepVGG-style reparameterization removes short skip connections at inference without an accuracy penalty.
    Section 3.5: justifies the RepSparseConv layers.
  • domain assumption INT8 quantization-aware training preserves sufficient accuracy for the detection task.
    Section 3.1: the choice of INT8 rests on this standard assumption in embedded DNN deployment.
  • domain assumption 2D cell processing is preferable to 3D voxel processing on FPGAs due to on-chip memory limits for Im2Col buffers.
    Section 3.1: a buffer-size calculation (52483 cells vs 1283 cells) motivates restricting the design to 2D cells.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LiFT: Lightweight, FPGA-tailored 3D object detection based on LiDAR data." pith.science (2026). https://pith.science/paper/TFV4QCQP

@misc{pith2026250111159,
  author       = {Pith},
  title        = {Pith review of: LiFT: Lightweight, FPGA-tailored 3D object detection based on LiDAR data},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TFV4QCQP}},
  note         = {Machine review of arXiv:2501.11159}
}
read the original abstract

This paper presents LiFT, a lightweight, fully quantized 3D object detection algorithm for LiDAR data, optimized for real-time inference on FPGA platforms. Through an in-depth analysis of FPGA-specific limitations, we identify a set of FPGA-induced constraints that shape the algorithm's design. These include a computational complexity limit of 30 GMACs (billion multiply-accumulate operations), INT8 quantization for weights and activations, 2D cell-based processing instead of 3D voxels, and minimal use of skip connections. To meet these constraints while maximizing performance, LiFT combines novel mechanisms with state-of-the-art techniques such as reparameterizable convolutions and fully sparse architecture. Key innovations include the Dual-bound Pillar Feature Net, which boosts performance without increasing complexity, and an efficient scheme for INT8 quantization of input features. With a computational cost of just 20.73 GMACs, LiFT stands out as one of the few algorithms targeting minimal-complexity 3D object detection. Among comparable methods, LiFT ranks first, achieving an mAP of 51.84% and an NDS of 61.01% on the challenging NuScenes validation dataset. The code will be available at https://github.com/vision-agh/lift.

Figures

Figures reproduced from arXiv: 2501.11159 by the authors.

Figure 1
Figure 1. An outlook on Dual-Bound Pillar Feature Net (DBPFN) structure [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 2
Figure 2. An outlook on LiFT structure where resolutionX = 2−8 ∗ (Xmax − Xmin), X denotes the location of a given point along the X-axis, and Xmin and Xmax define the boundaries of the point cloud along the same axis. In a similar manner, the features Ycoarse, Ydetail, Zcoarse, and Zdetail are defined. 3.5 LiFT design The schematic of our proposed architecture – LiFT – is shown in [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Reparametrisable convolution structure during training (on the left) and [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

16 extracted references · 12 canonical work pages

  1. [1]

    AMD/Xilinx:Vitisaimodelzoo.https://xilinx.github.io/Vitis-AI/3.0/html/docs/ workflow-model-zoo.html (Last access 22th November 2024)

    work page 2024

  2. [2]

    In: Applied Reconfigurable Computing

    Brum,H.,Véstias,M.,Neto,H.:Lidar3dobjectdetectioninfpgawithlowbitwidth quantization. In: Applied Reconfigurable Computing. Architectures, Tools, and Applications. pp. 90–105. Springer Nature Switzerland, Cham (2024)

    work page 2024

  3. [3]

    arXiv preprint arXiv:1903.11027 (2019)

    Caesar,H.,Bankiti,V.,Lang,A.H.,Vora,S.,etal.:nuscenes:Amultimodaldataset for autonomous driving. arXiv preprint arXiv:1903.11027 (2019)

    arXiv 2019

  4. [4]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Chen, Y., Liu, J., Zhang, X., Qi, X., et al.: Voxelnext: Fully sparse voxelnet for 3d object detection and tracking. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 21674–21683 (June 2023)

    work page 2023

  5. [5]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Ding, X., Zhang, X., Ma, N., Han, J., et al.: Repvgg: Making vgg-style convnets great again. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 13733–13742 (June 2021)

    work page 2021

  6. [6]

    International Journal of Robotics Research (IJRR) (2013)

    Geiger, A., Lenz, P., Stiller, C., Urtasun, R.: Vision meets robotics: The kitti dataset. International Journal of Robotics Research (IJRR) (2013)

    2013

  7. [7]

    In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Lang, A.H., Vora, S., Caesar, H., et al., L.Z.: Pointpillars: Fast encoders for object detection from point clouds. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 12689–12697 (June 2019)

    work page 2019

  8. [8]

    In: 2023 IEEE Intelligent Vehicles Symposium (IV)

    Latotzke, C., Kloeker, A., Schoening, S., Kemper, F., et al.: Fpga-based acceler- ation of lidar point cloud processing and detection on the edge. In: 2023 IEEE Intelligent Vehicles Symposium (IV). pp. 1–8 (2023)

    work page 2023

Show all 16 references
  1. [9]

    In: 2024 IEEE International Sympo- sium on High-Performance Computer Architecture (HPCA)

    Lee, M., Park, S., Kim, H., Yoon, M., et al.: Spade: Sparse pillar-based 3d object detection accelerator for autonomous driving. In: 2024 IEEE International Sympo- sium on High-Performance Computer Architecture (HPCA). pp. 454–467 (2024)

    2024

  2. [10]

    In: 2022 IEEE 40th International Conference on Computer Design (ICCD)

    Li, X., Ren, A., Tan, Y., Li, X., et al.: Vea: An fpga-based voxel encoding accelera- tor for 3d object detection with lidar. In: 2022 IEEE 40th International Conference on Computer Design (ICCD). pp. 509–516 (2022)

    2022

  3. [11]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops

    Park, S., Lee, M., Choi, J., Choi, J.: Selectively dilated convolution for accuracy- preserving sparse pillar-based embedded 3d object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops. pp. 8104–8113 (June 2024)

    2024

  4. [12]

    In: Computer Vision – ECCV 2022

    Shi, G., Li, R., Ma, C.: Pillarnet: Real-time and high-performance pillar-based 3d object detection. In: Computer Vision – ECCV 2022. pp. 35–52. Springer Nature Switzerland, Cham (2022)

    2022

  5. [13]

    Journal of Signal Processing Systems (2021)

    Stanisz, J., Lis, K., Gorgon, M.: Implementation of the pointpillars network for 3d object detection in reprogrammable heterogeneous devices using finn. Journal of Signal Processing Systems (2021)

    2021

  6. [14]

    Sun, P., Kretzschmar, H., Dotiwalla, X., Chouard, A., et al.: Scalability in percep- tion for autonomous driving: Waymo open dataset (2019)

    2019

  7. [15]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Yin, T., Zhou, X., Krahenbuhl, P.: Center-based 3d object detection and tracking. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 11784–11793 (June 2021)

    2021

  8. [16]

    Zhou, S., Tian, Z., Chu, X., Zhang, X., et al.: Fastpillars: A deployment-friendly pillar-based 3d detector (2023), https://arxiv.org/abs/2302.02367

    2023 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.