REVIEW 4 major objections 5 minor 30 references
FastCaps: A Design Methodology for Accelerating Capsule Network on Field Programmable Gate Arrays
T0 review · 4 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash
Pith's one-line read This paper proposes a two-step method to put a complete Capsule Network on a low-cost FPGA and reports the first full-fledged CapsNet deployment of its kind, reaching 1351 FPS on MNIST and 934 FPS on F-MNIST.
desk verdict A credible CapsNet FPGA acceleration result with real measured speedups, but the accuracy-drop claim is inconsistent and the Taylor softmax approximation needs bounding or end-to-end accuracy evidence. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
Two mechanisms carry the argument. (1) Look-Ahead Kernel Pruning (LAKP) scores each convolution kernel by summing per-parameter look-ahead scores—each weight's magnitude times the Frobenius norms of the adjacent-layer weight slices it connects to—and removes the lowest-scoring kernels layer by layer. This structured pruning collapses the PrimaryCaps capsule count from 1152 to 252 (MNIST) and 432 (F-MNIST), reducing routing weight parameters by 1280x. (2) The simplified routing datapath replaces exp() with a fixed five-term Taylor polynomial around a=0.5 and computes division a/b as exp(log(a)-log(b)), then reorders loops and uses a 10-PE array to parallelize the agreement and fully-connected
What would settle it
Instrument the pruned MNIST and F-MNIST models to record the actual routing logits, then compare exact softmax output against the five-term Taylor softmax. If any test sample yields a different argmax digit, or if the measured accuracy gap between the two versions exceeds the claimed <1 percent, the no-drop claim would be refuted.
Extended reading notes
Core claim
The central claim is that a full CapsNet—not just the routing loop—can fit on an FPGA and run fast if you attack both the parameter count and the routing math. CapsNets are usually considered too large and too irregular for FPGAs; the paper's counter is a two-part pipeline. LAKP turns simple magnitude-based kernel pruning into structured pruning by summing a look-ahead score per kernel, cutting the digit-capsule routing computation drastically and reducing routing weight parameters by up to 1280x. Then the routing algorithm's exp() and div() are replaced with Taylor approximations and multiplications, cutting softmax latency by 85 percent, after which loop reordering makes the agreement and
Load-bearing premise
The routing softmax stays accurate when the fixed five-term Taylor expansion of exp() covers the range of routing logits seen during inference; the paper reports no accuracy drop but never bounds that range.
Editorial extensions
If this is right
- Pruning the layer before a capsule layer directly cuts the number of capsules, so any CapsNet with a convolution front-end can inherit this acceleration without changing the routing algorithm's semantics.
- The routing optimizations are modular: other capsule networks deployed on FPGA or ASIC could adopt the Taylor-based exp/div blocks and reordered loops to gain an 85 percent reduction in softmax latency without a full redesign.
- Because LAKP prunes whole kernels, the accelerator stores only kernel indices—about 0.1 percent of the surviving weight count—rather than per-weight indices, keeping on-chip memory small.
- LAKP transfers beyond CapsNet: on VGG-19 and ResNet-18 across CIFAR-10 and GTSRB it consistently outperforms magnitude-based kernel pruning, with the largest gains in the high-sparsity regime.
- With 16-bit quantization and the simplified routing math, the design reportedly keeps accuracy loss under 1 percent, making the pruned-and-optimized model a practical candidate for edge deployments.
Reading between the lines
- The claimed <1 percent accuracy drop couples the pruning step with the Taylor softmax approximation; a natural follow-up is to measure routing-logit ranges on other datasets, since a fixed five-term expansion may need more terms or a range check when routing dynamics differ from MNIST/F-MNIST.
- The reported 270x and 187x speedups combine two effects—pruning and math simplification—and the paper already separates them in throughput (82/48 FPS after pruning, 1351/934 after optimization); a reader should note that accuracy is reported after pruning, while the additional routing approximation is asserted not to change it.
- Because the routing simplification is independent of training, the same accelerator design could be re-targeted to higher-resolution inputs or deeper capsule stacks, with the practical limit set by BRAM and DSP headroom on larger FPGAs.
- On low-cost boards, throughputs in the hundreds-to-thousands of FPS range would make CapsNet competitive with lightweight CNNs for small-image classification workloads, an implication the authors leave implicit.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes FastCaps, a two-step methodology for deploying a full CapsNet on a low-cost FPGA (Xilinx PYNQ-Z1). First, LAKP (Look-Ahead Kernel Pruning) is introduced as a structured kernel-pruning method that uses look-ahead scores summed per kernel. Second, the dynamic routing algorithm is simplified by approximating exp() and div() with Taylor-series-based operations, reordering loops, and parallelizing operations with PE arrays. The paper reports throughput improvements from 5 FPS for the original CapsNet to 1351 FPS (MNIST) and 934 FPS (F-MNIST) for the pruned and optimized design, with effective compression rates of 99.26% and 98.84%, respectively, and claims an accuracy drop of less than 1%. It also presents LAKP results on VGG-19 and ResNet-18. The claimed contributions are a first full-fledged CapsNet FPGA accelerator, a novel pruning methodology, and a hardware-oriented routing optimization.
Significance. If the accuracy claims are validated, this is a practically useful contribution: it demonstrates that a full CapsNet can be mapped to a low-cost FPGA with very high throughput and high compression, using a pruning method that produces regular sparse structures compatible with hardware. The concrete hardware measurements (throughput, resource utilization, latency) are a strength. The paper also provides empirical comparisons of LAKP against magnitude-based kernel pruning on three architectures and multiple datasets, which supports the pruning methodology. However, the paper does not provide end-to-end accuracy measurements of the deployed FPGA design, and one of the stated accuracy claims is contradicted by the paper's own Table I. These issues make the central claim (high-speed FPGA CapsNet with <1% accuracy drop) currently unsubstantiated.
major comments (4)
- [Table I / Section V] The conclusion states LAKP achieves 99.26% and 98.84% compression 'with an accuracy drop of less than 1%'. For F-MNIST, Table I gives a baseline test error of 10.31%; at 1.37% survived weights the LAKP error is 11.82%, i.e. +1.51 percentage points. The claimed 98.84% compression corresponds to 1.16% survived weights; interpolating between the 1.37% and 0.25% rows yields an error around 12.4%, about +2.1 pp. The <1% claim is therefore contradicted by the paper's own data. Please either restrict the claim to MNIST, report the actual drop for each dataset, or adjust pruning rates so the error increase is within the stated bound.
- [Section III-B, Eq. (2)-(3)] The 'without dropping accuracy' assertion for the Taylor approximation of exp() is unsupported. The manuscript does not bound the range of routing logits b_ij observed during inference, nor does it report the approximation error of Eq. (2) over that range. The polynomial is not globally accurate: for negative logits with magnitude greater than about 1.7 it can return negative values, which breaks softmax and makes Eq. (3) ill-defined because log of a negative operand is not real. Dynamic routing updates b_ij by accumulating dot-product agreements, so such logits are plausible. As written, Eq. (2) also is not the Taylor expansion around a=0.5: the coefficients do not match and the powers should be of (x-a), not x. Please provide the empirical distribution of b_ij, an error bound for Eq. (2) on that range, and end-to-end accuracy of the FPGA implementation.
- [Section IV-B] No accuracy measurement of the deployed hardware is reported. The paper states that the 16-bit quantization 'did not lead to a reduction in accuracy', but no accuracy numbers for the FPGA model are given. Because the deployed model combines pruning, 16-bit quantization, and the approximate softmax/squash of Section III-B, the central claim of '<1% accuracy drop' cannot be checked without the accuracy of the complete FPGA design. Please include test accuracy of the full bitstream on MNIST and F-MNIST, compared to the baseline software CapsNet.
- [Section I and Related Work] The paper claims to be the first to accelerate a full-fledged CapsNet on FPGA, but the related work includes [30], 'Enabling Capsule Networks at the Edge through Approximate Softmax and Squash Operations' (ISLPED 2022), which is directly related to the routing simplification proposed here. A quantitative comparison with [30] (and with any other FPGA CapsNet works) is needed to substantiate the novelty claim. If [30] already implements a full CapsNet on FPGA, the 'first' statement must be qualified.
minor comments (5)
- [Section III-A] The acronym is inconsistently written as 'LKAP' in one sentence ('We propose a Look-Ahead Kernel Pruning (LKAP)') and as 'LAKP' elsewhere. Also, in Algorithm 1, 'fksi as a si-th smallest element' should read 'the s_i-th smallest element'.
- [Eq. (2)] The sentence 'where a = 0.5' is confusing because the polynomial is written in powers of x, not (x - a). Please clarify the expansion point and write the polynomial in the standard form (x - a).
- [Table III caption] Typo: 'CapNet' should be 'CapsNet'.
- [Fig. 1] The unit 'FPJ' appears twice; likely should be 'pJ' or 'fJ' for energy. Please correct.
- [Section IV-A / Fig. 5] Figure 5 is described as comparing 'various pruning techniques', but the axes and the quantities plotted are not defined. Please add axis labels and clarify whether the y-axis is test error and the x-axis is survived rate.
Circularity Check
No significant circularity: the reported FPS, compression, and accuracy are empirical measurements on hardware and benchmarks; self-citations do not provide load-bearing evidence.
full rationale
The paper's central claims are hardware measurements (throughput, latency, resource utilization) and empirical pruning comparisons on held-out test sets, not derivations from the claims themselves. LAKP is benchmarked against magnitude-based kernel pruning [14] on CapsNet, VGG-19, and ResNet-18, and the pruning results are reported as test errors on standard datasets. The routing optimizations (Taylor-series exp and log-based division) are hardware approximations whose latency effects are measured in clock cycles and FPS; they are not fitted to the final accuracy claim. The many self-citations to Shafique's prior work appear in related work and do not supply the evidence for the paper's measured results. The assertion that the Taylor approximation causes 'no drop in accuracy' is not substantiated by an end-to-end accuracy measurement, and Table I's F-MNIST row (1.37% survived: 11.82% error vs. 10.31% baseline) is inconsistent with the '<1% drop' claim; however, this is a correctness/validation concern, not a circular derivation. No equation or pruning step is defined in terms of the quantity it is used to predict, and no fitted parameter is relabeled as a prediction.
Assumptions & free parameters
free parameters (4)
- Per-layer sparsity rates s_i
- Taylor expansion point a=0.5 and truncation order (5 terms) =
a=0.5, 5 terms
- PE array size (10 PEs) =
10
- Loop parallelization factor 'fact'
assumptions (3)
- standard math Taylor series expansion of exp() (Eq. 2) and the identity a/b = exp(log a - log b) (Eq. 3)
- domain assumption Routing logits bij remain within the range where the truncated Taylor expansion of exp() is accurate
- domain assumption Kernel pruning of the Convolution and PrimaryCaps layers preserves CapsNet accuracy after fine-tuning
Cite this review
Pith. "Pith review of FastCaps: A Design Methodology for Accelerating Capsule Network on Field Programmable Gate Arrays." pith.science (2026). https://pith.science/paper/MP7L6SGX
@misc{pith2026250903103,
author = {Pith},
title = {Pith review of: FastCaps: A Design Methodology for Accelerating Capsule Network on Field Programmable Gate Arrays},
year = {2026},
howpublished = {\url{https://pith.science/paper/MP7L6SGX}},
note = {Machine review of arXiv:2509.03103}
}
read the original abstract
Capsule Network (CapsNet) has shown significant improvement in understanding the variation in images along with better generalization ability compared to traditional Convolutional Neural Network (CNN). CapsNet preserves spatial relationship among extracted features and apply dynamic routing to efficiently learn the internal connections between capsules. However, due to the capsule structure and the complexity of the routing mechanism, it is non-trivial to accelerate CapsNet performance in its original form on Field Programmable Gate Array (FPGA). Most of the existing works on CapsNet have achieved limited acceleration as they implement only the dynamic routing algorithm on FPGA, while considering all the processing steps synergistically is important for real-world applications of Capsule Networks. Towards this, we propose a novel two-step approach that deploys a full-fledged CapsNet on FPGA. First, we prune the network using a novel Look-Ahead Kernel Pruning (LAKP) methodology that uses the sum of look-ahead scores of the model parameters. Next, we simplify the nonlinear operations, reorder loops, and parallelize operations of the routing algorithm to reduce CapsNet hardware complexity. To the best of our knowledge, this is the first work accelerating a full-fledged CapsNet on FPGA. Experimental results on the MNIST and F-MNIST datasets (typical in Capsule Network community) show that the proposed LAKP approach achieves an effective compression rate of 99.26% and 98.84%, and achieves a throughput of 82 FPS and 48 FPS on Xilinx PYNQ-Z1 FPGA, respectively. Furthermore, reducing the hardware complexity of the routing algorithm increases the throughput to 1351 FPS and 934 FPS respectively. As corroborated by our results, this work enables highly performance-efficient deployment of CapsNets on low-cost FPGA that are popular in modern edge devices.
Figures
Figures from the paper (10 more)
Reference graph
Works this paper leans on
-
[30]
Enabling Capsule Networks at the Edge through Approximate Softmax and Squash Operations,
A. Marchisio, B. Bussolino, E. Salvati, M. Martina, G. Masera, M. Shafique, “Enabling Capsule Networks at the Edge through Approximate Softmax and Squash Operations,” in ISLPED, 2022
work page 2022
-
[16]
Enabling Highly Efficient Capsule Networks Processing Through Software-Hardware Co- Design,
X. Zhang, X.Fu, D. Zhuang, C. Xie and S. L. Song, “Enabling Highly Efficient Capsule Networks Processing Through Software-Hardware Co- Design,” in IEEE Transactions on Computers , 2021
work page 2021
-
[1]
Imagenet classification with deep convolutional neural networks,
A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in Advances in Neural Infor- mation Processing Systems , 2012
work page 2012
-
[2]
R-FCN: Object detection via region based fully convolutional networks,
J. Dai, Y . Li, K. He, and J. Sun, “R-FCN: Object detection via region based fully convolutional networks,” in International Conference on Neural Information Processing Systems , 2016
work page 2016
-
[3]
Learning a classification model for segmentation,
X. Ren and J. Malik, “Learning a classification model for segmentation,” in IEEE International Conference on Computer Vision , 2003
work page 2003
-
[4]
Dynamic Routing Between Capsules,
S. Sabour, N. Frosst and G. E. Hinton, “Dynamic Routing Between Capsules,” in International Conference on Neural Information Process- ing Systems , 2017
work page 2017
-
[5]
Deep residual learning for image recognition,
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in IEEE Conference on Computer Vision and Pattern Recognitions, 2016
work page 2016
-
[6]
Imagenet classification with deep convolutional neural networks,
A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in Neural Information Pro- cessing Systems , 2012
work page 2012
Show all 30 references
-
[7]
CapsuleVOS: Semi-Supervised Video Object Segmentation Using Capsule Routing,
K. Duarte, Y . S. Rawat and M. Shah,“ CapsuleVOS: Semi-Supervised Video Object Segmentation Using Capsule Routing,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , 2019
2019
-
[8]
Accurate Automatic Glioma Segmentation in Brain MRI images Based on CapsNet,
M. J. Aziz, A. A. T. Zade, P. Farnia, M. Alimohamadi, B. Makkiabadi, A. Ahmadian and J. Alirezaie, “Accurate Automatic Glioma Segmentation in Brain MRI images Based on CapsNet,” in Annual International Conference of the IEEE Engineering in Medicine and Biology Society , 2021
2021
-
[9]
Novel Deep Learning Model for Traffic Sign Detection Using Capsule Networks,
A. D. Kumar, R. Karthika and L. Parameswaran, “Novel Deep Learning Model for Traffic Sign Detection Using Capsule Networks,” in Interna- tional Journal of Pure and Applied Mathematics , 2018
2018
-
[10]
Ristretto: A frame- work for empirical study of resource-efficient inference in convolutional neural networks,
P. Gysel, J. Pimentel, M Motamedi and S. Ghiasi, “Ristretto: A frame- work for empirical study of resource-efficient inference in convolutional neural networks,” in IEEE Transactions on Neural Networks and Learn- ing Systems , 2018
2018
-
[11]
Convolutional neural net- works with low-rank regularization,
C. Tai, T. Xiao, Y . Zhang, X. Wang and W. E, “Convolutional neural net- works with low-rank regularization,” https://arxiv.org/abs/1511.06067, 2016
2016 arXiv
-
[12]
An Exploration of Parameter Redundancy in Deep Networks With Circulant Projections,
Y . Cheng, F. X. Yu, R. S. Feris, S. Kumar, A. Choudhary and S. Chang, “An Exploration of Parameter Redundancy in Deep Networks With Circulant Projections,” in IEEE International Conference on Computer Vision, 2015
2015
-
[13]
Optimal Brain Damage,
Y . L. Cun, J. S. Denker and S. A. Sol1a, “Optimal Brain Damage,” in Advances in Neural Information Processing Systems , 1989
1989
-
[14]
Exploring the Regularity of Sparse Structure in Convolutional Neural Networks,
H. Mao, S. Han, J. Pool, W. Li, X. Liu, Y . Wang and W. J. Dally, “Exploring the Regularity of Sparse Structure in Convolutional Neural Networks,” in Neural Information Processing Systems , 2017
2017
-
[15]
Lookahead: A Far-sighted Al- ternative of Magnitude-based Pruning,
S. Park, J. Lee, S. Mo and J. Shin, “Lookahead: A Far-sighted Al- ternative of Magnitude-based Pruning,” in International Conference on Learning Representations, 2020
2020
-
[17]
DESCNet: Developing Efficient Scratchpad Memories for Capsule Network Hard- ware,
A. Marchisio, V . Mrazek, M. A. Hanif and M. Shafique, “DESCNet: Developing Efficient Scratchpad Memories for Capsule Network Hard- ware,” in IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems , 2021
2021
-
[18]
FEECA: Design Space Exploration for Low-Latency and Energy-Efficient Cap- sule Network Accelerators,
A. Marchisio, V . Mrazek, M. A. Hanif and M. Shafique, “FEECA: Design Space Exploration for Low-Latency and Energy-Efficient Cap- sule Network Accelerators,” in IEEE Transactions on V ery Large Scale Integration (VLSI) Systems , 2021
2021
-
[19]
Eyeriss: An Energy- Efficient Reconfigurable Accelerator for Deep Convolutional Neural Networks,
Y . Chen, T. Krishna, J. S. Emer and V . Sze, “Eyeriss: An Energy- Efficient Reconfigurable Accelerator for Deep Convolutional Neural Networks,” in IEEE Journal of Solid-State Circuits , 2016
2016
-
[20]
Angel-Eye: A Complete Design Flow for Mapping CNN Onto Embedded FPGA,
K. Guo, L. Sui, J. Qiu, J. Yu, J. Wang, S. Yao, S. Han, Y . Wang and H. Yang, “Angel-Eye: A Complete Design Flow for Mapping CNN Onto Embedded FPGA,” in IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems , 2017
2017
-
[21]
Learning both weights and connections for efficient neural networks,
S. Han, J. Pool, J. Tran, and W. J. Dally, “ Learning both weights and connections for efficient neural networks,” in Advances in Neural Information Processing Systems , 2015
2015
-
[22]
Hessian-Aware Pruning and Optimal Neural Implant,
S. Yu, Z. Yao, A. Gholami, Z. Dong, S. Kim, M. W. Mahoney and K. Keutzer, “Hessian-Aware Pruning and Optimal Neural Implant,” in IEEE/CVF Winter Conference on Applications of Computer Vision , 2022
2022
-
[23]
Movement Pruning: Adaptive Sparsity by Fine-Tuning,
V . Sanh, T. Wolf and A. M. Rush, “Movement Pruning: Adaptive Sparsity by Fine-Tuning,” in Neural Information Processing Systems , 2020
2020
-
[24]
PrunedCaps: A Case For Pri- mary Capsules Discrimination,
R. Sharifi, P. Shiri and A. Baniasadi, “PrunedCaps: A Case For Pri- mary Capsules Discrimination,” in International Conference on Machine Learning and Applications , 2021
2021
-
[25]
Q-CapsNets: A Specialized Framework for Quantizing Capsule Networks,
A. Marchisio, B. Bussolino, A. Colucci, M. Martina, G. Masera and M. Shafique, “Q-CapsNets: A Specialized Framework for Quantizing Capsule Networks,” in Design Automation Conference , 2020
2020
-
[26]
Very deep convolutional networks for large-scale image recognition,
K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” in International Conference on Learning Representations, 2015
2015
-
[27]
Hardware and Software Optimizations for Accelerating Deep Neural Networks: Survey of Current Trends, Challenges, and the Road Ahead,
M. Capra, B. Bussolino, A. Marchisio, G. Masera, M. Martina, M. Shafique, “Hardware and Software Optimizations for Accelerating Deep Neural Networks: Survey of Current Trends, Challenges, and the Road Ahead,” in IEEE Access8: 225134-225180 , 2020
2020
-
[28]
An Updated Survey of Efficient Hardware Architectures for Accelerating Deep Convolutional Neural Networks,
M. Capra, B. Bussolino, A. Marchisio, M. Shafique, G. Masera, M. Martina, “An Updated Survey of Efficient Hardware Architectures for Accelerating Deep Convolutional Neural Networks,” in Future Internet 12(7): 113 , 2020
2020
-
[29]
RoHNAS: A Neural Architecture Search Framework With Conjoint Optimization for Adversarial Robustness and Hardware Effi- ciency of Convolutional and Capsule Networks,
A. Marchisio, V . Mrazek, A. Massa, B. Bussolino, M. Martina, M. Shafique, “RoHNAS: A Neural Architecture Search Framework With Conjoint Optimization for Adversarial Robustness and Hardware Effi- ciency of Convolutional and Capsule Networks,” in IEEE Access 10: 109043-109055, 2022
2022
Reviewed August 5, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.