REVIEW 4 major objections 6 minor 2 cited by
Comparative Analysis of FPGA and GPU Performance for Machine Learning-Based Track Reconstruction at LHCb
T0 review · 4 major / 6 minor · reviewed 2026-08-09 · deepseek-v4-flash
Pith's one-line read An 8-bit FPGA implementation of an ML track-finding step matches a GPU at roughly 60% of the power.
desk verdict Useful, honest engineering benchmark, but the headline FPGA-vs-GPU comparison is a synthesis extrapolation, not a measured result. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the embedding multilayer perceptron of the ETX4VELO GNN pipeline: a fully connected feedforward network with a 3-dimensional input, three hidden layers of 8 neurons with ReLU activations, and a 3-dimensional output. On the FPGA side, the machinery is HLS4ML, which converts the trained PyTorch model into HLS code, synthesizes it to Verilog/VHDL through Vivado HLS, and deploys it as a PYNQ overlay with AXI-Stream communication. The FPGA model is quantized to fixed-point precision, ap_fixed<16,6> on the PYNQ-Z2 and ap_fixed<8,3> on the Alveo boards, while the GPU baseline uses INT8 quantization through TensorRT. The performance projection for the Alveo U250 is carried by the assumption that the post-synthesis resource usage of a single IP core scales linearly, allowing roughly 205 parallel cores on the board.
What would settle it
Measure the actual throughput of the 8-bit embedding MLP on an Alveo U250 with about 205 parallel IP cores and realistic AXI-Stream event input; if it falls below the GPU's 0.82 million events per second, or the power draw exceeds roughly 230 W under load, the paper's headline comparison does not hold.
Extended reading notes
Core claim
The paper's central claim is that the 8-bit Alveo U250 implementation of the embedding MLP is on par with the GeForce RTX 3090 implementation, with the potential to slightly outperform it, while consuming just over 60% of the power used by the GPU. This claim rests on a Vivado post-synthesis latency estimate of 85 ns and the assumption that about 205 copies of the IP core can run in parallel on the U250, giving a theoretical throughput of 1.10 million events per second versus 0.82 million for the GPU. The smaller Alveo U50 would reach about 0.55 million events per second, slightly slower than the GPU but using almost five times less power. The paper also reports energy per event of 210 µJ for the U250 and 430 µJ for the GPU, and notes the trade-off that the U250 costs roughly ten times more than the GPU.
Load-bearing premise
The projected U250 throughput assumes that one IP core's post-synthesis latency and resource usage scale linearly to about 205 parallel cores with no degradation from data movement, memory bandwidth, or routing congestion, and no Alveo board measurement is shown.
Editorial extensions
If this is right
- If the projected U250 throughput holds, FPGAs can match the current GPU step of LHCb's first-level trigger at lower power, making them a candidate for moving pattern recognition onto data-acquisition boards.
- The HLS4ML workflow lets researchers without FPGA expertise produce low-latency inference firmware, potentially widening the use of FPGAs in high-energy physics.
- The energy-per-event figures, 210 µJ for the U250 versus 430 µJ for the GPU, imply meaningfully lower operating power, though the U250's upfront cost is roughly ten times the GPU's.
- The Alveo U50 offers a nearly fivefold power reduction at slightly lower throughput, giving an intermediate option when purchase cost dominates.
- Future optimizations such as quantization-aware training could reduce precision loss and improve the physics performance of the FPGA inference.
Reading between the lines
- The paper's power comparison uses the boards' maximum rated power even though the 8-bit implementation uses under 1% of the U250's LUTs; an actual implementation might draw less power, which would strengthen the FPGA advantage, or more if I/O and many parallel cores consume heavily.
- The 205-core extrapolation assumes no degradation from AXI-Stream data movement, memory bandwidth, or routing congestion; a hardware measurement with many parallel cores would be the direct test.
- The same HLS4ML workflow could be applied to later steps of the GNN pipeline, but the embedding MLP is small, and larger models may not show the same linear resource scaling.
- The cost comparison excludes maintenance, development, and upgrade costs; including them could shift the break-even time beyond the estimated six years for the U50.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper compares the throughput and power efficiency of an MLP-based track-reconstruction step for LHCb's VELO, implemented on FPGAs via HLS4ML, against an existing GPU (RTX 3090) implementation. The FPGA work includes a measured deployment on a PYNQ-Z2 board (16-bit, 1.2M inferences/s) and theoretical projections for Alveo U50 and U250 boards (8-bit) based on Vivado post-synthesis latency (85 ns) and linear resource scaling to ~103 and ~205 parallel IP cores. The resulting projected throughputs are 0.55M and 1.10M events/s for the U50 and U250, compared with 0.82M events/s measured on the GPU. Power usage is taken from rated maximums, yielding energy-per-event estimates and an operating-cost comparison. The paper concludes that the U250 could match or slightly outperform the GPU while consuming about 60% of the power.
Significance. If the projected Alveo throughput were confirmed on hardware, the paper would provide a useful data point for HLS4ML-based FPGA inference as an alternative to GPUs in high-throughput HEP trigger applications. The paper is transparent about the theoretical nature of the Alveo numbers in Section III.B and provides a reproducible synthesis workflow, including a resource table (Table I) from Vivado. The measured PYNQ-Z2 result is a genuine hardware benchmark, and the GPU throughput is taken from the existing measured Allen pipeline. However, the central comparison is not a hardware benchmark in the usual sense: the headline Alveo numbers are extrapolations, not measurements, and the model is a small MLP (3-8-8-8-3), so the significance for the full ETX4VELO GNN pipeline is indirect. The paper does not provide code or data, but it specifies the workflow in sufficient detail to be reproduced.
major comments (4)
- [Section III.B, Table II] The central throughput comparison rests entirely on a theoretical extrapolation: a single-IP post-synthesis latency of 85 ns and a linear scaling to about 205 IP cores on the U250, with no Alveo hardware measurement reported. The only measured end-to-end HLS4ML/PYNQ deployment in the paper (Section III.A.6) achieves 1.2 million inferences per second on the PYNQ-Z2, i.e., about 833 ns per inference, which is an order of magnitude larger than the 85 ns used for the U250. The paper does not demonstrate that the AXI-Stream/processing-system overhead that apparently dominates the PYNQ measurement disappears on the Alveo cards, nor that 205 cores can be fed concurrently without I/O or memory-bandwidth bottlenecks. Therefore the statement in Section III.B that the U250 'is on par' with the RTX 3090 is not supported by the data presented; at most it is an optimistic projection. Please provide actual Alveo hardware measurements, or, if they are not available, reframe the abstract, introduction, and conclusion to state explicitly that the comparison is a theoretical estimate rather than a benchmarked result.
- [Section III.B, Table I] The linear resource extrapolation from one IP using 8,396 LUTs to 205 IPs on the U250 is unrealistic because 205 IPs would consume 1,721,180 of the 1,728,000 LUTs (99.6%), leaving essentially no LUTs for the AXI interconnect, control logic, or routing. Place and route would very likely fail at such utilization. The paper's assumption that 'resource usage is not going to change dramatically' as more IPs are packed is not validated; in practice, multiple instances typically require additional routing resources and may not achieve the same clock frequency. This extrapolation is the basis for the 1.10 million events/s figure and should be supported by a multi-IP synthesis test, or the claim should be removed.
- [Section III.B, power comparison] The power comparison uses maximum rated board power for the Alveo cards (75 W and 225/230 W) and the GPU (350 W) rather than measured power draw. Since the 8-bit implementation uses less than 1% of LUTs (Table I), the actual FPGA power draw could be far below the rated maximum. While this assumption is conservative for the paper's energy-efficiency claim, it makes the claim 'consuming just over 60% of the power' an upper bound, not a measured result. Please state explicitly that all power values are rated maximums, not measured, and harmonize the U250 value (225 W in the text, 230 W in Table II).
- [Section III.A.7] The accuracy of the 8-bit FPGA implementation used in the comparison is not reported. The paper states that for the 16-bit <16,6> implementation, 97% of values are predicted within 10% of the correct values, but no analogous validation is given for the <8,3> implementation. Since the GPU comparison uses TensorRT INT8 with calibration, the two implementations may have different numerical precision; a throughput comparison is only meaningful if both meet the required physics performance. Please add an accuracy or physics-performance check for the <8,3> implementation, and clarify the relationship between ap_fixed<8,3> and INT8 (the former has 5 fractional bits, the latter is integer).
minor comments (6)
- [Section I] The acronym 'HSL4ML' appears once; it should be 'HLS4ML'.
- [Section III.B, references] The Alveo U50 price is quoted as 2965 USD with reference [21], which points to the U250 product page, while reference [23] points to the U50 page. The references appear to be swapped; please correct them.
- [Section III.A.2] The change from ap_fixed<16,6> on the PYNQ-Z2 to ap_fixed<8,3> on the Alveo cards is not motivated. Please explain why <8,3> is chosen and how it maps to the GPU's INT8 quantization.
- [Section II] Please state explicitly whether the quoted GPU throughput of 820k events/s includes only the embedding MLP or also any preceding graph-construction or data-preparation steps, so that the comparison with the FPGA MLP-only implementation is clearly apples-to-apples.
- [Table II] The U50 price is listed as 3000 USD in Table II while the text says 2965 USD; please make the values consistent.
- [Section III.B, cost analysis] The operating-cost calculation assumes a fixed electricity price of 125.2 EUR/MWh and continuous operation at maximum power for the whole year; a sentence noting the sensitivity of the payback-period estimate to these assumptions would improve robustness.
Circularity Check
No circular derivation: the FPGA throughput estimate is an explicit synthesis-based extrapolation, and the GPU baseline is an external measured result cited from prior work.
full rationale
Walking the derivation chain: the GPU throughput (820k events/s) is quoted from the authors' prior Allen implementation [15] as an Allen built-in timer measurement; it is an externally reproducible measurement, not a quantity defined by the present paper, so it does not count as circular under the stated rules. The PYNQ-Z2 figure (1.2M inferences/s) is a hardware measurement. The Alveo U250 figure is computed as 1/(85 ns) from a Vivado post-synthesis latency estimate, multiplied by floor(1,728,000/8,396) = 205 assumed parallel IPs and divided by 2200 hits/event, giving 1.1M events/s. Each step is arithmetic or a clearly stated extrapolation; no step defines its output in terms of the claimed conclusion, and no fitted parameter is renamed as a prediction. The power and energy-per-event numbers are TDP/throughput arithmetic with stated assumptions. The paper explicitly labels the Alveo numbers as estimates and theoretical maxima, which is an unsupported-extrapolation or correctness concern, not a circularity concern. The self-citations to [15] and [16] provide the GPU implementation, the sample, and the pipeline description; because those are measured, parameter-free, externally checkable results, they do not raise the circularity score under hard rule 4. No uniqueness theorem, ansatz-via-citation, or renaming of a known result is present. The central derivation is self-contained arithmetic grounded in synthesis reports and one measured deployment; the honest finding is no significant circularity.
Assumptions & free parameters
assumptions (3)
- domain assumption Vivado HLS synthesis reports accurately predict deployed latency and resource usage on Alveo FPGAs.
- domain assumption Resource usage scales linearly when instantiating up to 205 IP cores on the Alveo U250, with no additional overhead.
- domain assumption Official maximum power specifications of the Alveo cards represent actual power draw during inference.
Cite this review
Pith. "Pith review of Comparative Analysis of FPGA and GPU Performance for Machine Learning-Based Track Reconstruction at LHCb." pith.science (2026). https://pith.science/paper/AXS25MAA
@misc{pith2026250202304,
author = {Pith},
title = {Pith review of: Comparative Analysis of FPGA and GPU Performance for Machine Learning-Based Track Reconstruction at LHCb},
year = {2026},
howpublished = {\url{https://pith.science/paper/AXS25MAA}},
note = {Machine review of arXiv:2502.02304}
}
abstract
In high-energy physics, the increasing luminosity and detector granularity at the Large Hadron Collider are driving the need for more efficient data processing solutions. Machine Learning has emerged as a promising tool for reconstructing charged particle tracks, due to its potentially linear computational scaling with detector hits. The recent implementation of a graph neural network-based track reconstruction pipeline in the first level trigger of the LHCb experiment on GPUs serves as a platform for comparative studies between computational architectures in the context of high-energy physics. This paper presents a novel comparison of the throughput of ML model inference between FPGAs and GPUs, focusing on the first step of the track reconstruction pipeline$\unicode{x2013}$an implementation of a multilayer perceptron. Using HLS4ML for FPGA deployment, we benchmark its performance against the GPU implementation and demonstrate the potential of FPGAs for high-throughput, low-latency inference without the need for an expertise in FPGA development and while consuming significantly less power.
Forward citations
Cited by 2 Pith papers
-
Real-Time Analysis of Unstructured Data with Machine Learning on Heterogeneous Architectures
A graph neural network (ETX4VELO) reconstructs LHCb VELO tracks with performance comparable to the production 'search by triplet' algorithm while running end to end in the GPU-based first-level trigger, with additiona...
-
Review of Machine Learning for Real-Time Analysis at the Large Hadron Collider experiments ALICE, ATLAS, CMS and LHCb
A SMARTHEP-network review of deployed and developing machine-learning methods for real-time triggering at ALICE, ATLAS, CMS and LHCb, with examples of industrial crossover.
Reference graph
Works this paper leans on
-
[1]
Framework TDR for the LHCb Upgrade : Technical Design Report,
LHCb Collaboration, “Framework TDR for the LHCb Upgrade : Technical Design Report,” CERN, Tech. Rep., 2012. [Online] . Available: https://cds.cern.ch/record/1443882
arXiv 2012
-
[2]
LHCb VELO Upgrade Technical Design Report,
——, “LHCb VELO Upgrade Technical Design Report,” CERN, T ech. Rep., 2013. [Online]. Available: https://cds.cern.ch/re cord/1624070
-
[3]
LHCb Upgrade GPU High Level Trigger Technical Desig n Report,
——, “LHCb Upgrade GPU High Level Trigger Technical Desig n Report,” 2020. [Online]. Available: https://cds.cern.ch /record/2717938
-
[4]
Can FPGAs Beat GPUs in Accelerating Next-Generation Deep Neural Networks?
E. Nurvitadhi et al. , “Can FPGAs Beat GPUs in Accelerating Next-Generation Deep Neural Networks?” in Proceedings of the 2017 ACM/SIGDA International Symposium on Field-Programm able Gate Arrays , ser. FPGA ’17. New Y ork, NY , USA: Association for Computing Machinery, Feb. 2017, pp. 5–14. [Online]. Ava ilable: https://doi.org/10.1145/3020078.3021740
arXiv 2017
-
[5]
Throughput-Optimized OpenCL-based FPGA Accelerator for Large-Scale Convolutional Neural Network s,
N. Suda et al. , “Throughput-Optimized OpenCL-based FPGA Accelerator for Large-Scale Convolutional Neural Network s,” in Proceedings of the 2016 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays , ser. FPGA ’16. New Y ork, NY , USA: Association for Computing Machinery, Feb. 2016, pp. 16 –25. [Online]. Available: https://doi.org/10.1145/2...
-
[6]
Boosted Decision Trees in the Level-1 Muon Endcap Trigger at CMS,
D. E. Acosta et al. , “Boosted Decision Trees in the Level-1 Muon Endcap Trigger at CMS,” 2017. [Online]. Available: https:/ /cds.cern.ch/ record/2290188
-
[7]
S. Giagu, “Fast and Resource-Efficient Deep Neural Netwo rk on FPGA for the Phase-II Level-0 Muon Barrel Trigger of the A TLAS Exp eri- ment,” EPJ W eb Conf., vol. 245, p. 01021, 2020
work page 2020
-
[8]
LHCb Collaboration, “Physics Case for an LHCb Upgrade II - Opportunities in Flavour Physics, and Beyond, in the HL-LHC Era,” CERN, Tech. Rep., 2016. [Online]. Available: https://cds. cern.ch/record/ 2636441
work page 2016
Show all 25 references
-
[9]
Fast Inference of Deep Neural Networks in FPGAs for Particle Physics,
J. Duarte et al. , “Fast Inference of Deep Neural Networks in FPGAs for Particle Physics,” Journal of Instrumentation , vol. 13, no. 07, pp. P07 027–P07 027, Jul. 2018. [Online]. Available: http://ar xiv.org/abs/ 1804.06913
2018 arXiv
-
[10]
FastML Team, “HLS4ML,” Jul. 2024. [Online]. Available : https:// github.com/fastmachinelearning/hls4ml
2024
-
[11]
Keras: Deep Learning for Humans
Keras developers, “Keras: Deep Learning for Humans.” [ Online]. Available: https://keras.io/
-
[12]
TensorFlow: A System for Large-Scale Machine Learning,
M. Abadi et al. , “TensorFlow: A System for Large-Scale Machine Learning,” Proceedings of the 12th USENIX conference on Operating Systems Design and Implementation , pp. 265–283, Nov. 2016
2016
-
[13]
PyTorch: An Imperative Style, High-Performance Deep Learning Library,
A. Paszke et al. , “PyTorch: An Imperative Style, High-Performance Deep Learning Library,” in Proceedings of the 33rd International Conference on Neural Information Processing Systems . Red Hook, NY , USA: Curran Associates Inc., Dec. 2019, pp. 8026–8037
2019
-
[14]
Performance of a Geometric Deep Learning Pipeline for HL-LHC Particle Tracking,
X. Ju et al. , “Performance of a Geometric Deep Learning Pipeline for HL-LHC Particle Tracking,” The European Physical Journal C , vol. 81, no. 10, p. 876, Oct. 2021. [Online]. Available: http s://doi.org/ 10.1140/epjc/s10052-021-09675-8
2021 doi
-
[15]
Graph Neural Network-Based Track Finding in the LHCb V erte x Detector,
A. Correia, F. I. Giasemis, N. Garroum, V . V . Gligorov, a nd B. Granado, “Graph Neural Network-Based Track Finding in the LHCb V erte x Detector,” Journal of Instrumentation , vol. 19, no. 12, p. P12022, Dec
-
[16]
Graph Neural Network-Based Pipeline for Track Fin ding in the VELO at LHCb,
——, “Graph Neural Network-Based Pipeline for Track Fin ding in the VELO at LHCb,” in Connecting The Dots 2023 (CTD 2023) , Oct. 2023, pp. PROC–CTD2023–34. [Online]. Available: https:// arxiv.org/ abs/2406.12869
2023
-
[17]
NVIDIA TensorRT,
TensorRT developers, “NVIDIA TensorRT,” 2024. [Onlin e]. Available: https://developer.nvidia.com/tensorrt
2024
-
[18]
PyTorch-Quantization: Training and Eval- uating PyTorch Models with Simulated Quantization
NVIDIA developers, “PyTorch-Quantization: Training and Eval- uating PyTorch Models with Simulated Quantization.” [On- line]. Available: https://docs.nvidia.com/deeplearnin g/tensorrt/pytorch- quantization- toolkit/docs/
-
[19]
PYNQ - Python Productivity to AMD Adap tive Compute Platforms
AMD developers, “PYNQ - Python Productivity to AMD Adap tive Compute Platforms.” [Online]. Available: http://www.pyn q.io/
-
[20]
Vivado HLS
——, “Vivado HLS.” [Online]. Available: https://www.a md.com/en/ products/software/adaptive- socs-and-fpgas/vivado.html
-
[21]
Alveo U250 Data Center Accelerator Card
——, “Alveo U250 Data Center Accelerator Card.” [Online ]. Available: https://www.amd.com/en/products/accelerators/alveo/u250/a- u250-a64g-pq-g.html
-
[22]
NVIDIA GeForce RTX 3090 Specifications,
TechPowerUp, “NVIDIA GeForce RTX 3090 Specifications, ” Jan. 2025. [Online]. Available: https://www.techpowerup.com/gpu- specs/geforce- rtx-3090.c3622
2025
-
[23]
Alveo U50 Data Center Accelerator Car d
AMD developers, “Alveo U50 Data Center Accelerator Car d.” [Online]. Available: https://www.amd.com/en/products/accelerat ors/alveo/u50/a- u50-p00g-pq-g.html
-
[24]
Energy Dashboard
Swiss Federal Office of Energy SFOE, “Energy Dashboard. ” [Online]. Available: https://www.dashboardenergie.admin.ch/preise/strom-karte
-
[2024]
Available: https://dx.doi.org/10.1088/ 1748-0221/19/12/ P12022
[Online]. Available: https://dx.doi.org/10.1088/ 1748-0221/19/12/ P12022
Reviewed August 9, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.