REVIEW 3 major objections 6 minor 33 references
Hardware Accelerators for Artificial Intelligence
T0 review · 3 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read This chapter argues that specialized hardware accelerators are necessary because the von Neumann bottleneck makes data movement the dominant cost in AI workloads, and it maps the resulting design space from GPUs and FPGAs to ASICs…
desk verdict A competent but unoriginal survey chapter whose central von-Neumann-bottleneck framing is asserted rather than demonstrated; useful for newcomers, not for researchers. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The organizing lens is the multiply-accumulate (MAC) operation treated as the canonical AI workload, together with the cost of moving the data needed for it. Each surveyed architecture is essentially a strategy for feeding MAC arrays: systolic arrays and crossbars maximize parallel matrix-vector throughput, near-memory and in-memory designs shorten the distance data travels, on-chip buffers and compressed formats such as compressed sparse column and run-length coding cut redundant transfers, and reduced-precision arithmetic shrinks both storage and energy per operation.
What would settle it
A controlled benchmark that runs the same DNN inference workload on a state-of-the-art GPU with fully optimized, data-reuse-aware software and on a processing-in-memory accelerator, comparing energy per multiply-accumulate, would test the premise: if the GPU matches or beats PIM once data movement is hidden, the claim that the von Neumann bottleneck dominates collapses.
Extended reading notes
Core claim
The paper's central claim is that specialized hardware accelerators are necessary, not optional, for modern AI: the von Neumann architecture's separation of memory and compute creates a bottleneck that CPUs cannot resolve and GPUs only partially mask, so the field has moved to designs that keep data close to arithmetic units, exploit massive parallelism, and accept reduced precision. On that basis it classifies accelerators by trade-off: GPUs are a high-throughput stepping stone with heavy energy costs, FPGAs are reconfigurable low-latency inference engines, ASICs such as the TPU and the DianNao family are maximally efficient but inflexible, neuromorphic chips are brain-inspired and still early, and memory-centric and processing-in-memory designs such as PRIME, Neurocube, and NVM-based accelerators are the direct answer to the data-movement problem. The chapter further claims that the winning designs co-optimize hardware with the algorithm, exploiting sparsity, compressing activations, and adjusting precision, rather than simply adding more compute.
Load-bearing premise
The load-bearing premise is that moving data between memory and processing units is the principal cost in AI computation, so the von Neumann architecture is the real bottleneck; if algorithmic efficiency or raw compute density were the limiting factor, the surveyed designs' strengths would shift.
Editorial extensions
If this is right
- If data movement is the dominant cost, memory-centric and processing-in-memory accelerators should keep gaining on CPUs and GPUs for matrix-heavy inference workloads.
- Exploiting sparsity and compression, as in Eyeriss v2 and CompAct, should make deep neural network inference practical on battery-powered mobile and edge devices.
- Reduced-precision arithmetic and lossy compression will continue to buy large efficiency gains wherever AI tasks tolerate small accuracy loss.
- No single accelerator class dominates, so heterogeneous systems that combine CPUs, GPUs, FPGAs, and ASICs are the likely route to both performance and flexibility.
- Neuromorphic and NVM-based accelerators remain viable long-term directions, but their device variability and manufacturing challenges must be solved before they can compete broadly.
Reading between the lines
- The chapter's own logic implies that algorithm-level changes such as pruning, quantization, and sparsity-aware scheduling may be as important as the underlying device, since several surveyed accelerators gain most of their advantage from avoiding or compressing data movement.
- A natural testable extension is to benchmark representative accelerators from each class on a fixed workload with data movement normalized; if the performance gap narrows, software optimization may substitute for specialized hardware.
- The emphasis on inference-oriented designs suggests that the next frontier may be training accelerators, where weight updates demand far more data movement and processing-in-memory advantages are less certain.
- If energy per operation becomes the dominant metric, the taxonomy points toward heterogeneous edge systems rather than homogeneous data-center GPU clusters.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This manuscript is a survey chapter on hardware accelerators for artificial intelligence. It motivates the need for specialized hardware by arguing that traditional CPUs and even GPUs are inadequate for AI workloads because of the von Neumann architecture, energy inefficiency, and limited parallelism (Section 1.1.2). It then presents a taxonomy of accelerator classes (GPUs, FPGAs, ASICs, neuromorphic ICs, emerging memory-centric devices) with comparative strengths and weaknesses (Section 1.2.3), reviews case studies including Google TPU, IBM TrueNorth/NorthPole, Intel Loihi, and memory-centric/PIM designs (Section 1.2.2), and describes specific accelerator architectures: NeuFlow, DianNao series, NPU, RENO, Neurocube, PRIME, TPU, Eyeriss v2, and CompAct (Sections 1.3.1-1.3.9). The chapter closes with design considerations, optimization techniques, applications, and future trends. No original derivation or novel experimental result is presented; the contribution is a broad, referenced overview intended for newcomers and practitioners.
Significance. If taken as a pedagogical survey, the chapter has value: it collects a wide range of well-known accelerator architectures and organizes them by category, with figures recreated from the primary sources and a substantial reference list. The coverage of memory-centric and processing-in-memory designs in Sections 1.2.2 and 1.3.5-1.3.6 is reasonably comprehensive, and the discussion of design trade-offs in Section 1.4 is useful orientation. However, the chapter makes no original technical claim and therefore stands or falls on factual accuracy and internal consistency. As submitted, it contains a clearly incorrect hardware classification (H100 labeled an ASIC in Fig. 1.3), an unsupported and internally tense central premise about the von Neumann bottleneck (Section 1.1.2), and citation and organizational errors. These issues are fixable, but they currently undermine the chapter's reliability as a reference.
major comments (3)
- [Section 1.1.2] The central premise that the von Neumann architecture and data movement are the principal bottlenecks for AI workloads is asserted without quantitative support. The chapter provides no energy-per-access versus compute-energy comparison, no roofline analysis, and no citation establishing that memory access dominates for representative neural network layers. This premise is load-bearing because the subsequent emphasis on near-memory and processing-in-memory designs (Sections 1.2.2, 1.3.5, 1.3.6, 1.4.1) rests on the assumption that data movement is the dominant cost. Please either add quantitative grounding (e.g., citing established energy-per-access data or a roofline analysis for dense GEMM layers) or soften the claim to present data movement as one of several important bottlenecks alongside parallelism, reduced precision, and sparsity, which the chapter itself lists as independent motivations.
- [Sections 1.1.2 and 1.3.7] The classification of GPUs as 'adherent to the Von Neumann architecture' is loose and creates an internal tension. The TPU described in Section 1.3.7 and Fig. 1.10 also has a stored-program control unit and explicitly moves data through a systolic array; the differentiator between GPUs and TPUs is specialization of dataflow, precision, and control, not the absence of a von Neumann control model. This framing should be revised to avoid implying that GPUs are von Neumann while ASICs are not, which would strengthen the motivational argument and remove an inconsistency.
- [Fig. 1.3] The caption fragment around Fig. 1.3 labels the Nvidia H100 Hopper chip as 'NVIDIA H100 AI ASIC.' The H100 is a GPU, not an ASIC. This contradicts Section 1.2.2, which correctly places the H100 in the GPU category, and it undermines the taxonomy presented in Section 1.2.3. Please correct the label so the figure matches the surrounding text.
minor comments (6)
- [Section 1.2.3] The comparative analysis lists the categories in the order A, B, E, C, D; the Neuromorphic ICs subsection (E) appears before ASICs (C) and Emerging Devices (D). Please reorder the subsections to follow the stated sequence.
- [References] Reference entries [6]-[9] are never cited in the body of the chapter. They should either be cited where relevant (e.g., in a discussion of ML-based hardware design) or removed from the reference list.
- [Section 1.3.3] The text refers to 'Figure 6' but the figure is numbered Fig. 1.6; please make the cross-reference consistent.
- [Section 1.4.2] The heading '3 Emerging Technologies::' has a doubled colon, and the numbering scheme repeats '3' for both 'Hardware Co-design' and 'Emerging Technologies'.
- [Abstract] The phrase 'we aim to explore an in-depth exploration' is redundant; consider 'we provide an in-depth exploration' or similar.
- [Section 1.1.2] 'section1.3' should read 'Section 1.3' with a space.
Circularity Check
No circularity found: this is a descriptive survey whose claims about accelerator necessity, architecture, and comparative performance are attributed to external sources, with no fitted parameters, no derived predictions, and no self-citations.
full rationale
The chapter attempts no derivation chain; it is a survey that reports architectural descriptions and performance comparisons attributed to external prior work (e.g., Jouppi et al. for TPU, Chen et al. for DianNao, Chi et al. for PRIME, Farabet et al. for NeuFlow, Kim et al. for Neurocube, Zhang et al. for CompAct). None of the 33 references is authored by the chapter's own authors (Ahsan, Dhungel, Chowdhury, Hasan, Hoque), so no self-citation chain exists, and no uniqueness theorem or ansatz is imported from the authors' prior work. The only potentially load-bearing framing claim, Section 1.1.2's assertion that CPUs and GPUs are inadequate for AI due to the von Neumann architecture, is an empirical premise supported by citations [15, 16], not a quantity derived from the paper's own definitions; even if the premise were quantitatively under-supported, that is an evidence-strength or correctness concern, which the hard rules explicitly exclude from circularity. The chapter's conclusion that specialization helps is additionally supported by independent motivations it lists (parallelism, reduced precision, RISC-style instructions, sparsity), so it does not reduce to any single definitional premise. The skeptical observation that GPUs are called von Neumann-limited while the TPU (which also has a stored-program control unit) is praised is an internal consistency tension, not a circular reduction. A passage in Section 1.3.8 even notes that RLC 'is not explicitly mentioned in EyerissV2's architecture,' showing the authors distinguish between their own narrative and the cited source's content. Per hard rule 3, a paper whose content is self-contained against external benchmarks should receive a score of 0-2; this paper receives 0 because there is no circular step at all, only descriptive synthesis of externally attributed results.
Assumptions & free parameters
Cite this review
Pith. "Pith review of Hardware Accelerators for Artificial Intelligence." pith.science (2026). https://pith.science/paper/XRNLLVIZ
@misc{pith2026241113717,
author = {Pith},
title = {Pith review of: Hardware Accelerators for Artificial Intelligence},
year = {2026},
howpublished = {\url{https://pith.science/paper/XRNLLVIZ}},
note = {Machine review of arXiv:2411.13717}
}
read the original abstract
In this chapter, we aim to explore an in-depth exploration of the specialized hardware accelerators designed to enhance Artificial Intelligence (AI) applications, focusing on their necessity, development, and impact on the field of AI. It covers the transition from traditional computing systems to advanced AI-specific hardware, addressing the growing demands of AI algorithms and the inefficiencies of conventional architectures. The discussion extends to various types of accelerators, including GPUs, FPGAs, and ASICs, and their roles in optimizing AI workloads. Additionally, it touches on the challenges and considerations in designing and implementing these accelerators, along with future prospects in the evolution of AI hardware. This comprehensive overview aims to equip readers with a clear understanding of the current landscape and future directions in AI hardware development, making it accessible to both experts and newcomers to the field.
Figures
Reference graph
Works this paper leans on
-
[6]
Cunxi Yu, Houping Xiao, and Giovanni De Micheli. 2018. Developing synthesis flows with- out human knowledge. In Proceedings of the 55th Annual Design Automation Conference (DAC ’18). Association for Computing Machinery, New York, NY, USA, Article 50, 1 –6. https://doi.org/10.1145/3195970.3196026
arXiv 2018
-
[9]
Hung-Yi Liu and L. P. Carloni, ”On learning -based methods for design -space exploration with High-Level Synthesis,” 2013 50th ACM/EDAC/IEEE Design Automation Conference (DAC), Austin, TX, 2013, pp. 1 -7. keywords: Radio frequency;Training;Ground penetrat - ing radar;Approximation methods;Solid modeling;Discrete Fourier transforms;Prediction algorithms;Sy...
work page 2013
-
[1]
G. Sapunov, in TPU v2 was unveiled at Google I/O in May 2017 , https://moocaholic.medium.com/hardware-for-deep-learning-part-4-asic-96a542fe6a813948
work page 2017
-
[2]
NVIDIA, in NVIDIA Tesla P100: The World’s First AI Supercomputing Data Center GPU , https://www.nvidia.com/en-us/data-center/tesla-p100/
-
[3]
S. Shankland, in Nvidia H100 Hopper chip, https://www.cnet.com/pictures/see-nvidias-h100- hopper-chip-up-close/4/
-
[4]
Intel, in Intel Advances Neuromorphic with Loihi 2, New Lava Software Framework and New Partners , https://www.intel.com/content/www/us/en/newsroom/news/intel-unveils- neuromorphic-loihi-2-lava-software.htmlgs.4dc193
-
[5]
ALINX AX7Z020: SoC FPGA Development Board, https://www.xilinx.com/products/boards- and-kits/1-1bu8447.html
-
[7]
A Review of Machine Learning Techniques in Analog Integrated Circuit Design Automation
Mina, R.; Jabbour, C.; Sakr, G.E. A Review of Machine Learning Techniques in Analog Integrated Circuit Design Automation. Electronics 2022, 11, 435. https://doi.org/10.3390/electronics11030435
Show all 33 references
-
[8]
Kang Liu, Haoyu Yang, Yuzhe Ma, Benjamin Tan, Bei Yu, Evangeline F. Y. Young, Ramesh Karri, and Siddharth Garg. 2020. Adversarial Perturbation Attacks on ML-based CAD: A Case Study on CNN-based Lithographic Hotspot Detection. ACM Trans. Des. Autom. Electron. Syst. 25, 5, Artic...
2020
-
[10]
F. Akopyan et al., ”TrueNorth: Design and Tool Flow of a 65 mW 1 Million Neu- ron Programmable Neurosynaptic Chip,” in IEEE Transactions on Computer -Aided De - sign of Integrated Circuits and Systems, vol. 34, no. 10, pp. 1537 -1557, Oct. 2015, doi: 10.1109/TCAD.2015.2474396
2015
-
[11]
Modha et al
Dharmendra S. Modha et al. ,Neural inference at the fr ontier of energy, space, and time.Science382,329-335(2023).DOI:10.1126/science.adh1174
2023 doi
-
[12]
Zahoor, F., Hussin, F.A., Isyaku, U.B. et al. Resistive random access memory: introduction to device mechanism, materials and application to neuromorphic computing. Discover Nano 18, 36 (2023). https://doi.org/10.1186/s11671-023-03775-y
2023 doi
-
[13]
Manuel Le Gallo and Abu Sebastian 2020 J. Phys. D: Appl. Phys. 53 213002. DOI 10.1088/1361-6463/ab7794 References 37
2020 doi
-
[14]
Sze, Resistance random access memory,Materials Today,Volume 19, Issue 5,2016,Pages 254-264, ISSN 1369-7021, https://doi.org/10.1016/j.mattod.2015.11.009
Ting-Chang Chang, Kuan-Chang Chang, Tsung-Ming Tsai, Tian -Jian Chu, Simon M. Sze, Resistance random access memory,Materials Today,Volume 19, Issue 5,2016,Pages 254-264, ISSN 1369-7021, https://doi.org/10.1016/j.mattod.2015.11.009
2016 doi
-
[15]
An Updated Survey of Efficient Hardware Architectures for Accelerating Deep Convolutional Neural Net- works
Capra, M.; Bussolino, B.; Marchisio, A.; Shafique, M.; Masera, G.; Martina, M. An Updated Survey of Efficient Hardware Architectures for Accelerating Deep Convolutional Neural Net- works. Future Internet 2020, 12, 113. https://doi.org/10.3390/fi12070113
2020 doi
-
[16]
Nurvitadhi, D
E. Nurvitadhi, D. Sheffield, Jaewoong Sim, A. Mishra, G. Venkatesh and D. Marr, ”Acceler- ating Binarized Neural Networks: Comparison of FPGA, CPU, GPU, and ASIC,” 2016 Inter- national Conference on Field-Programmable Technology (FPT), Xi’an, China, 2016, pp. 77- 84, doi: 10.1...
2016
-
[17]
Esmaeilzadeh, A
H. Esmaeilzadeh, A. Sampson, L. Ceze and D. Burger, ”Neural Acceleration for General- Purpose Approximate Programs,” 2012 45th Annual IEEE/ACM Internat ional Sympo - sium on Microarchitecture, Vancouver, BC, Canada, 2012, pp. 449 -460, doi: 10.1109/MI - CRO.2012.48. keywords: ...
2012 doi
-
[18]
Farabet, B
C. Farabet, B. Martini, B. Corda, P. Akselrod, E. Culurciello and Y. Le- Cun, ”NeuFlow: A runtime reconfigurable dataflow processor for vision,” CVPR 2011 WORKSHOPS, Colorado Springs, CO, USA, 2011, pp. 109 -116, doi: 10.1109/CVPRW.2011.5981829. keywords: Tiles;Computer archit...
2011
-
[19]
X. Liu et al., ”RENO: A high-efficient reconfigurable neuromorphic computing accelerator design,” 2015 52nd A CM/EDAC/IEEE Design Automation Conference (DAC), San Fran - cisco, CA, USA, 2015, pp. 1-6, doi: 10.1145/2744769.2744900. keywords: Active appearance model;Artificial n...
2015
-
[20]
Tianshi Chen, Zidong Du, Ninghui Sun, Jia Wang, Chengyong Wu, Yunji Chen, and Olivier Temam. 2014. DianNao: a small -footprint high-throughput accelerator for ubiqui - tous machine -learning. SIGARCH Comput. Archit. News 42, 1 (March 2014), 269 –284. https://doi.org/10.1145/26...
2014
-
[21]
Z. Du et al., ”ShiDianNao: Shifting vision processing closer to the sensor,” 2015 ACM/IEEE 42nd Annual International Symposium on Computer Architecture (ISCA), Portland, OR, USA, 2015, pp. 92 -104, doi: 10.1145/2749469.2750389. keywords: Kernel;Neural net - works;Sensors;Energ...
2015
-
[22]
Daofu Liu, Tianshi Chen, Shaoli Liu, Jinhong Zhou, Shengyuan Zhou, Olivier Te- man, Xiaobing Feng, Xuehai Zhou, and Yunji Chen. 2015. PuDianNao: A Polyva- lent Machine Learning Accelerator. SIGPLAN Not. 50, 4 (April 2015), 369 –381. https://doi.org/10.1145/2775054.2694358
2015
-
[23]
Jouppi, N. P., Young, C., Patil, N., Patterson, D., Agrawal, G., Bajwa, R., Bates, S., Bhatia, S., Boden, N., Borchers, A., Boyle, R., Cantin, P., Chao, C., Clark, C., Coriell, J., Daley, M., Dau, M., Dean, J., Gelb, B., Yoon, D. H. (2017). In-Datacenter Performance Analysis o...
2017
-
[24]
D. Kim, J. Kung, S. Chai, S. Yalamanchili and S. Mukhopadhyay, ”Neurocube: A Pro- grammable Digital Neuromorphic Architecture with High -Density 3D Memory,” 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA), Seoul, Korea (South), 2016, pp. 380-3...
2016 doi
-
[25]
Jeff (Jun) Zhang, Parul Raj, Shuayb Zarar, Amol Ambardekar, and Siddharth Garg. 2019. CompAct: On-chip Compression of Activations for Low Power Systolic Array Based CNN Acceleration. ACM Trans. Embed. Comput. Syst. 18, 5s, Article 47 (October 2019), 24 pages. https://doi.org/1...
2019 doi
-
[26]
Y. -H. Chen, T. Krishna, J. S. Eme r and V. Sze, ”Eyeriss: An Energy -Efficient Reconfig- urable Accelerator for Deep Convolutional Neural Networks,” in IEEE Journal of Solid - State Circuits, vol. 52, no. 1, pp. 127 -138, Jan. 2017, doi: 10.1109/JSSC.2016.2616357. keywords: S...
2017
-
[27]
Y. -H. Chen, T. -J. Yang, J. Emer and V. Sze, ”Eyeriss v2: A Flexible Accelerator for Emerg- ing Deep Neural Networks on Mobile Devices,” in IEEE Journal on Emerging and Selected Topics in Circuits and Systems, vol. 9, no. 2, pp. 292 -308, June 2019, doi: 10.1109/JET - CAS.201...
2019
-
[28]
P. Chi et al., ”PRIME: A Novel Processing-in-Memory Architecture for Neural Net- work Computation in ReRAM -Based Main Memory,” 2016 ACM/IEEE 43rd Annual In - ternational Symposium on Computer Architecture (ISCA), Seoul, Korea (South), 2016, pp. 27 -39, doi: 10.1109/ISCA.2016....
2016 doi
-
[29]
X. Dong, C. Xu, Y. Xie and N. P. Jouppi, ”NVSim: A Circuit -Level Performance, Energy, and Area Model for Emerging Nonvolatile Memory,” in IEEE Transactions on Computer - Aided Design of Integrated Circuits and Systems, vol. 31, no. 7, pp. 994 -1007, July 2012, doi: 10.1109/TC...
2012
-
[30]
X. Peng, S. Huang, Y. Luo, X. Sun and S. Yu, ”DNN+NeuroSim: An End-to-End Benchmark- ing Framework for Compute-in-Memory Accelerators with Versatile Device Technologies,” 2019 IEEE International Electron Devices Meeting (IEDM), San Francisco, CA, USA, 2019, pp. 32.5.1-32.5.4, ...
2019
-
[31]
NVMExplorer: A Framework for Cross-Stack Com- parisons of Embedded Non-Volatile Memories
Pentecost, Lillian, et al. “NVMExplorer: A Framework for Cross-Stack Com- parisons of Embedded Non-Volatile Memories.” 2022 IEEE International Sympo- sium on High -Performance Computer Architecture (HPCA), IEEE, 2022. Crossref, https://doi.org/10.1109/hpca53966.2022.00073
2022
-
[32]
Compute-in-Memory Primitives for Energy- Efficient Machine Learning
Agrawal, Amogh (2021). Compute-in-Memory Primitives for Energy- Efficient Machine Learning. Purdue University Graduate School. Thesis. https://doi.org/10.25394/PGS.15048825.v1
2021 doi
-
[33]
V. Seshadri et al., ”Ambit: In-Memory Accelerator for Bulk Bitwise Operations Using Com- modity DRAM Technology,” 2017 50th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), Boston, MA, USA, 2017, pp. 273-287
2017
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.