Pith. sign in

REVIEW 4 major objections 4 minor 22 references

Hardware-Aware Feature Extraction Quantisation for Real-Time Visual Odometry on FPGA Platforms

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that the full, unmodified SuperPoint feature-extraction network can be quantised to 3 bits and compiled into an FPGA streaming pipeline that runs 640×480 images at 54 FPS, preserving the original architecture while…

desk verdict Solid FPGA quantisation study with a genuine 54 FPS full-architecture SuperPoint result, but the abstract and one table sentence overstate quality at low precision. read the letter →

arxiv 2507.07903 v1 pith:IR2KNOVG submitted 2025-07-10 cs.CV eess.IV

Mateusz Wasala , Mateusz Smolarczyk , Michal Danilowicz , Tomasz Kryjak This is my paper · ORCID
classification cs.CVeess.IV
keywords SuperPointvisualodometryfeatureextractionFPGASoCquantisation-awaretrainingFINNlow-bitquantisationhardwareacceleration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Visual odometry for robots and drones needs fast, accurate feature-point extraction, and the SuperPoint neural network is accurate but computationally heavy. This paper claims that the full, unmodified SuperPoint architecture can run in real time on an FPGA System-on-Chip if its weights and activations are quantised with quantisation-aware training, rather than by simplifying the network. Using the FINN compiler, the authors turn a 3-bit quantised SuperPoint model into a streaming hardware pipeline that processes 640×480 images at 54 FPS on a ZCU102 board and 27 FPS on the smaller Kria KV260, faster than prior FPGA accelerators that used 8-bit DPUs or trimmed architectures. On HPatches detector metrics the quantised models stay in the same range as the floating-point baseline, while on TUM visual-odometry trajectories lower precision monotonically increases pose error: rotation APE grows from 7.61 degrees at FP32 to 12.24 degrees at 3-bit and 19.97 degrees for the mixed 4-2-4 variant. The paper's core tradeoff is therefore real-time throughput at very low precision against a measurable loss in trajectory rotation accuracy.

What carries the argument

The load-bearing mechanism is the thresholding compilation step: FINN reorders the graph so every floating-point affine operation $ax+b$ preceding an activation requantisation is folded into the threshold list, replacing runtime arithmetic with stored thresholds $t_i \leftarrow (t_i-b)/a$, and the hardware returns the index of the first threshold exceeded. This is what converts the quantised network into a look-up-and-compare datapath rather than a multiply-accumulate engine, which is why the design runs at 300 MHz with only 8 DSPs. The central object preserved throughout is the complete SuperPoint network: shared encoder, feature-point decoder with Softmax, reshape and NMS, and descriptor decoder with interpolation and L2 normalisation, so any accuracy difference between the INT8, INT4, INT3, and mixed 4-2-4 variants is attributed to quantisation rather than to a changed topology. The FPGA implementation is a streaming pipeline with sliding-window convolution units, threshold modules whose stored tables dominate BRAM usage, and DMA transfers that stream weights from the processing system into programmable logic.

What would settle it

Run the INT3 FINN accelerator inside a complete SLAM system and compare absolute trajectory error against the 8-bit DPU implementation [8] and MobileSP [9] on identical TUM sequences; the central quality claim fails if the 3-bit model's trajectory error is not comparable to those 8-bit systems. A minimal check already available in Table V: require the 3-bit variant to keep APE rotation within 10% of the FP32 baseline, and it fails at 12.24 degrees versus 7.61 degrees.

Watch

Extended reading notes

Core claim

The discovery is that architectural fidelity and aggressive quantisation are compatible on FPGAs: keeping every layer of the original SuperPoint network, the authors quantise it to 3 bits with uniform affine quantisation (per-channel weight scales, zero-point zero, activation requantisation after each convolution) and compile it with FINN into a streaming, pipelined design. The compiler's key transformation re-expresses floating-point affine operations from quantisation or batch normalisation as threshold comparisons, so the programmable-logic datapath needs almost no DSPs: 8 as opposed to 1,307 for the DPU configuration. On the ZCU102 the 3-bit design runs 54 FPS at 640×480 with an estimated 85.3 mJ per frame, and the detector metrics on HPatches (repeatability 0.522, localisation error 1.34 px, homography estimation 0.75) are competitive with prior accelerators that simplified the architecture. The visual-odometry evaluation on TUM, however, shows the cost of that precision: absolute rotation error is 12.24 degrees at 3-bit versus 7.61 degrees for the FP32 baseline, and the mixed 4-2-4 precision variant performs worst at 19.97 degrees.

Load-bearing premise

The central claim that 3-bit quantisation preserves high detection quality depends on accepting the HPatches detector metrics as the measure of quality; if instead one takes the TUM visual-odometry trajectory errors as the measure, the 3-bit model's rotation error is 60% worse than the floating-point baseline and the claim is not supported.

Editorial extensions

If this is right

  • At 54 FPS and 85.3 mJ per frame on the ZCU102, the full SuperPoint front end meets the 20–25 FPS real-time bar for embedded visual odometry without architectural simplification.
  • Because the FINN datapath uses 8 DSPs versus 1,307 for the DPU configuration, the approach can target FPGA parts with very few multiplier blocks.
  • The DPU Softmax experiments show that a single operation's placement changes the speed-quality balance: moving Softmax to hardware raises throughput by about 18% but cuts the number of SuperGlue matches by roughly 65% at 640×480.
  • The precision ladder INT8→INT4→INT3→4-2-4 is monotonic in trajectory error, giving a direct calibration of how many bits a VO system can give up before rotation accuracy is seriously compromised.
  • Mixed 4-2-4 precision is worse than uniform 3-bit on every TUM trajectory metric reported, indicating that a simple mixed-precision heuristic can underperform uniform low-bit quantisation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: a uniform INT4 FINN design, which the paper did not build, would likely be the practical operating point for systems that care about trajectory accuracy, since INT4 keeps rotation APE at 10.41 degrees (37% above FP32) at a resource footprint similar to the 3-bit variant.
  • Beyond the paper: because threshold tables dominate BRAM, combining this threshold-based datapath with structured pruning of the descriptor decoder could reduce memory pressure further and unlock the 54 FPS design on boards smaller than the ZCU102.
  • Beyond the paper: the divergence between HPatches detector metrics and TUM trajectory errors suggests that low-bit quantisation studies of feature extractors should report both detector-level and full-pipeline metrics; HPatches alone would have supported INT3, while TUM alone would have pushed the choice back to INT8.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The manuscript studies quantisation of the SuperPoint feature-extraction network for FPGA-based visual odometry. The authors train quantised variants (INT8/INT4/INT3 and a mixed 4-2-4 scheme) with Brevitas, evaluate detector/descriptor quality on HPatches, and implement the 3-bit network on Kria KV260 and ZCU102 boards using FINN, reporting 27 and 54 FPS at 640x480 while preserving the full SuperPoint architecture. A Vitis AI DPU implementation is also evaluated, including the effect of placing Softmax on the DPU versus on the processor. Trajectory-level accuracy is measured on a TUM RGB-D sequence for all precision variants.

Significance. The paper provides a useful engineering data point: a full SuperPoint network can run in real time at sub-8-bit precision on an FPGA, using publicly available tools (Brevitas/FINN), with measured end-to-end FPS and resource usage. Its strengths include comparisons with three published FPGA accelerators, evaluation on the external HPatches benchmark, and a direct visual-odometry evaluation in Table V that partially exposes the accuracy cost of low precision. The central claim, however, needs to be re-scoped: the abstract's unqualified 'preserving high detection quality' and 'outperforming state-of-the-art' statements are not fully supported by the paper's own Table V and by the repeatability values in Table I.

major comments (4)
  1. [Section V.B, Table I] The sentence 'the Repeatability metric achieves higher values than in the other works analysed' is not correct for the 3-bit and mixed-precision variants: INT3 repeatability is 0.522 and mixed 4-2-4 is 0.50, both below the TCASI'22 value of 0.53. The preceding sentence that repeatability and localisation error 'remained largely unaffected' across all quantisation levels also understates the drop from 0.574 (FP32) to 0.522 (INT3) and 0.50 (mixed). Please correct these claims and qualify them by precision level.
  2. [Abstract and Section V.E, Table V] The abstract claims the implementation preserves 'high detection quality' and 'outperforming state-of-the-art solutions' without qualification. However, the paper's own TUM visual-odometry results show INT3 increasing APE rotation from 7.61 degrees (FP32) to 12.24 degrees (+60%) and mixed 4-2-4 to 19.97 degrees (+160%); the conclusion's description of the drop as 'slight' is not supported by these numbers. Since visual odometry is the target application, trajectory-level accuracy is a relevant arbiter of the central quality claim. The paper should either qualify the claim to detector-level metrics or add a substantive discussion of why HPatches performance is the appropriate measure despite the VO degradation.
  3. [Section V.E] The visual-odometry evaluation appears to be based on a single TUM sequence: no sequence identifier is given, no number of trials is reported, and no error bars or variability measures are provided. Because the quality claim depends on these numbers, the authors should specify the sequence(s) used and report results on additional sequences or at least state the variability across the TUM RGB-D benchmark before drawing conclusions about the practical acceptability of the degradation.
  4. [Table IV and Contributions] The 'outperforming state-of-the-art solutions' claim is based on FPS comparisons across different FPGA devices and implementations (e.g., TCASI'22 achieves 42 FPS on a ZCU104, while the proposed FINN design achieves 54 FPS on a ZCU102). This claim should be narrowed to 'highest reported frame rate at 640x480 for a full-architecture SuperPoint accelerator' or be accompanied by a same-board comparison or a more explicit discussion of the accuracy/resource trade-offs, so that it does not imply overall superiority.
minor comments (4)
  1. [Section IV.C] The threshold set is written as 'T = t0, t1, ....tn}' with a missing opening brace; the sentence 'The activation function and the activation requantification operation' should also use a consistent spelling of 'requantisation'.
  2. [Table II caption] The caption contains a duplicated and incomplete phrase: 'on the set of TUM and KITTI depending on usage, depending on the location of the Softmax operation.' Please rephrase to read cleanly.
  3. [Section IV.C] The sentence 'However, its role of PS crucial' is missing a verb; it should read 'its role in the PS is crucial'.
  4. [Figure 4] The axes of Figure 4 are labelled only with units ('[%]', '[MB]'); please clarify what 'Accuracy' and 'Complexity' represent and how the values are computed, since the relation between bitwidth and the plotted quantities is otherwise unclear.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: all quality and performance claims are evaluated against external datasets and baselines, with no fitted parameter renamed as a prediction.

full rationale

This paper makes no derivation claim that could support circularity. It takes an established architecture (SuperPoint [4]) and applies standard quantisation-aware training with Brevitas and FINN. Bit-widths (INT8, INT4, INT3, and mixed 4-2-4) are design hyperparameters, not quantities fitted to the evaluation targets. Detector quality results (Table I) are measured on the external HPatches benchmark; visual odometry results (Table V) are measured on the external TUM RGB-D benchmark; throughput, resource usage, and power data (Table IV) are measured on FPGA implementations or quoted from third-party papers. The comparisons against FCCM'20, TCASI'22, and APCCAS'24 are external baselines, not the authors' own prior work. There is no self-citation chain, no imported uniqueness theorem, and no ansatz smuggled in via citation. FINN and Brevitas are cited as tools, and the claims rest on measured experiments rather than on any fitted relationship. The abstract's 'outperforming state-of-the-art' claim is a speed and resource comparison, not a circular derivation. The possible overstatement about detection quality at 3-bit precision is a correctness or scope concern, not a circularity concern: the paper's own conclusion states that 'achieving high accuracy and precise feature point detection remains challenging at such low precision,' which is an honest admission rather than a circular step. Therefore the circularity score is 0.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new entities. Its central claims rest on design choices (bit widths, parallelism) and on benchmark assumptions; the main unverified inputs are the vendor-supplied resource/power estimates.

free parameters (3)
  • Quantisation bit widths for network variants = 8, 4, 3, and mixed 4-2-4 bits
    Chosen by hand to explore the accuracy/efficiency trade-off; the whole evaluation is organised around these choices.
  • Mixed-precision layer assignment = 4-bit in first conv, last ReLU, and last conv of each decoder; 2-bit elsewhere
    Ad hoc selection intended to preserve accuracy at input/output while compressing the middle of the network.
  • FINN parallelism variant = Two variants for Kria KV260 and ZCU102
    Design choice matching each board's resources; affects FPS and resource use.
assumptions (3)
  • domain assumption SuperPoint pretrained weights and the Homographic Adaptation pseudo-labeling procedure produce suitable training data for QAT.
    Invoked in Section IV.A; the validity of transfer from MS-COCO pseudo-labels to TUM/KITTI/HPatches is assumed.
  • domain assumption HPatches repeatability, localisation error, and homography estimation are valid proxies for feature extraction quality in VO.
    Used to claim competitive quality in Section V.B, while the VO results in Table V show large trajectory errors that complicate this interpretation.
  • domain assumption Resource utilisation and power estimates from AMD documentation and Vivado Power Analysis are representative of the deployed system.
    Stated in Table IV notes; these are not measured on hardware.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Hardware-Aware Feature Extraction Quantisation for Real-Time Visual Odometry on FPGA Platforms." pith.science (2026). https://pith.science/paper/IR2KNOVG

@misc{pith2026250707903,
  author       = {Pith},
  title        = {Pith review of: Hardware-Aware Feature Extraction Quantisation for Real-Time Visual Odometry on FPGA Platforms},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IR2KNOVG}},
  note         = {Machine review of arXiv:2507.07903}
}
read the original abstract

Accurate position estimation is essential for modern navigation systems deployed in autonomous platforms, including ground vehicles, marine vessels, and aerial drones. In this context, Visual Simultaneous Localisation and Mapping (VSLAM) - which includes Visual Odometry - relies heavily on the reliable extraction of salient feature points from the visual input data. In this work, we propose an embedded implementation of an unsupervised architecture capable of detecting and describing feature points. It is based on a quantised SuperPoint convolutional neural network. Our objective is to minimise the computational demands of the model while preserving high detection quality, thus facilitating efficient deployment on platforms with limited resources, such as mobile or embedded systems. We implemented the solution on an FPGA System-on-Chip (SoC) platform, specifically the AMD/Xilinx Zynq UltraScale+, where we evaluated the performance of Deep Learning Processing Units (DPUs) and we also used the Brevitas library and the FINN framework to perform model quantisation and hardware-aware optimisation. This allowed us to process 640 x 480 pixel images at up to 54 fps on an FPGA platform, outperforming state-of-the-art solutions in the field. We conducted experiments on the TUM dataset to demonstrate and discuss the impact of different quantisation techniques on the accuracy and performance of the model in a visual odometry task.

Figures

Figures reproduced from arXiv: 2507.07903 by the authors.

Figure 1
Figure 1. Schemes representing the main elements of the proposed method: a) Convolution neural network architecture called [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Comparison of estimated camera trajectories obtained [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Comparison of estimated orientation angles (roll, pitch, yaw) with reference values (ground truth) for the trajectory [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Trade-off between feature point detection accuracy and [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

22 extracted references · 20 canonical work pages

  1. [1]

    Distinctive image features from scale-invariant keypoints,

    D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” International Journal of Computer Vision , vol. 60, no. 2, pp. 91–110, November 2004

    work page 2004

  2. [2]

    Surf: Speeded up robust features,

    H. Bay, T. Tuytelaars, and L. Van Gool, “Surf: Speeded up robust features,” in Computer Vision–ECCV 2006: 9th European Conference on Computer Vision, Graz, Austria, May 7-13, 2006. Proceedings, Part I 9. Springer, 2006, pp. 404–417

    work page 2006

  3. [3]

    Orb: An efficient alternative to sift or surf,

    E. Rublee, V . Rabaud, K. Konolidge, and G. Bradski, “Orb: An efficient alternative to sift or surf,” in Proceedings of the IEEE International Conference on Computer Vision (ICCV) , November 2011, pp. 2564– 2571

    work page 2011

  4. [4]

    Superpoint: Self- supervised interest point detection and description,

    D. DeTone, T. Malisiewicz, and A. Rabinovich, “Superpoint: Self- supervised interest point detection and description,” in Proceedings of the IEEE conference on computer vision and pattern recognition workshops, 2018, pp. 224–236

    work page 2018

  5. [5]

    Superglue: Learning feature matching with graph neural networks,

    P.-E. Sarlin, D. DeTone, T. Malisiewicz, and A. Rabinovich, “Superglue: Learning feature matching with graph neural networks,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion (CVPR), June 2020, pp. 4938–4947

    work page 2020

  6. [6]

    Learning to match features with seeded graph matching network,

    H. Chen, Z. Luo, J. Zhang, L. Zhou, X. Bai, Z. Hu, C.-L. Tai, and L. Quan, “Learning to match features with seeded graph matching network,” in Proceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 6301–6310

    work page 2021

  7. [7]

    LoFTR: Detector-free local feature matching with transformers,

    J. Sun, Z. Shen, Y . Wang, H. Bao, and X. Zhou, “LoFTR: Detector-free local feature matching with transformers,” CVPR, 2021

    work page 2021

  8. [8]

    Cnn-based feature- point extraction for real-time visual slam on embedded fpga,

    Z. Xu, J. Yu, C. Yu, H. Shen, Y . Wang, and H. Yang, “Cnn-based feature- point extraction for real-time visual slam on embedded fpga,” in 2020 IEEE 28th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM) , 2020, pp. 33–37

    work page 2020

Show all 22 references
  1. [9]

    Mobilesp: An fpga-based real-time keypoint extraction hardware accel- erator for mobile vslam,

    Y . Liu, J. Li, K. Huang, X. Li, X. Qi, L. Chang, Y . Long, and J. Zhou, “Mobilesp: An fpga-based real-time keypoint extraction hardware accel- erator for mobile vslam,” IEEE Transactions on Circuits and Systems I: Regular Papers, vol. 69, no. 12, pp. 4919–4929, 2022

    2022

  2. [10]

    A low- hardware-overhead, high-energy-efficiency, and end-to-end cnn-based feature extraction accelerator for mobile visual slam,

    Z. Yin, B. Liu, J. Wang, Z. Shen, G. Li, and C. Wang, “A low- hardware-overhead, high-energy-efficiency, and end-to-end cnn-based feature extraction accelerator for mobile visual slam,” in 2024 IEEE Asia Pacific Conference on Circuits and Systems (APCCAS) , 2024, pp. 115–119

    2024

  3. [11]

    A benchmark for the evaluation of rgb-d slam systems,

    J. Sturm, N. Engelhard, F. Endres, W. Burgard, and D. Cremers, “A benchmark for the evaluation of rgb-d slam systems,” in Proc. of the International Conference on Intelligent Robot Systems (IROS) , Oct. 2012

    2012

  4. [12]

    Methods to evaluate accuracy-energy trade-off in operator- level approximate computing,

    B. Barrois, “Methods to evaluate accuracy-energy trade-off in operator- level approximate computing,” Theses, Université de Rennes, Dec

  5. [13]

    Xilinx/brevitas,

    G. Franco, A. Pappalardo, and N. J. Fraser, “Xilinx/brevitas,” 2025. [Online]. Available: https://doi.org/10.5281/zenodo.3333552

    2025 doi

  6. [15]

    Microsoft coco: Common objects in context,

    T.-Y . Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, “Microsoft coco: Common objects in context,” in ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceedings . Springer, 2014, pp. 740–755

    2014

  7. [16]

    Available: http://arxiv.org/abs/1712.05877

    [Online]. Available: http://arxiv.org/abs/1712.05877

    arXiv

  8. [17]

    QONNX: Representing Arbitrary-Precision Quantized Neural Networks,

    A. Pappalardo, Y . Umuroglu, M. Blott, J. Mitrevski, B. Hawks, N. Tran, V . Loncar, S. Summers, H. Borras, J. Muhizi, M. Trahms, S.-C. H. Hsu, S. Hauck, and J. Duarte, “QONNX: Representing Arbitrary-Precision Quantized Neural Networks,” in 4th Workshop on Accelerated Machine L...

    2022

  9. [18]

    Finn: A framework for fast, scalable binarized neural network inference,

    Y . Umuroglu, N. J. Fraser, G. Gambardella, M. Blott, P. Leong, M. Jahre, and K. Vissers, “Finn: A framework for fast, scalable binarized neural network inference,” in Proceedings of the 2017 ACM/SIGDA Interna- tional Symposium on Field-Programmable Gate Arrays , ser. FPGA ’17...

    2017

  10. [19]

    Finn-r: An end-to-end deep-learning framework for fast exploration of quantized neural networks,

    M. Blott, T. Preusser, N. Fraser, G. Gambardella, K. O’Brien, and Y . Umuroglu, “Finn-r: An end-to-end deep-learning framework for fast exploration of quantized neural networks,” 2018. [Online]. Available: https://arxiv.org/abs/1809.04570

    2018 arXiv

  11. [20]

    fastmachinelearning/qonnx,

    Q. authors, “fastmachinelearning/qonnx,” 06 2022. [Online]. Available: https://github.com/fastmachinelearning/qonnx

    2022

  12. [21]

    DPU Resource Utilization,

    AMD, “DPU Resource Utilization,” https://docs.amd.com/r/en-US/ pg338-dpu/Resource-Utilization, 2023, accessed: 2025-05-31

    2023

  13. [22]

    Hpatches: A benchmark and evaluation of handcrafted and learned local descriptors,

    V . Balntas, K. Lenc, A. Vedaldi, and K. Mikolajczyk, “Hpatches: A benchmark and evaluation of handcrafted and learned local descriptors,” in CVPR, 2017

    2017

  14. [2017]

    Available: https://theses.hal.science/tel-01665015

    [Online]. Available: https://theses.hal.science/tel-01665015

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.