REVIEW 3 major objections 5 minor 43 references
A Novel Design of Adaptive and Hierarchical Convolutional Neural Networks using Partial Reconfiguration on FPGA
T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read An adaptive CNN with a confidence-based gate preserves deep-model accuracy on embedded hardware while running as little as 44% of the computation.
desk verdict Real FPGA partial-reconfiguration work undercut by an internal SVHN accuracy contradiction and borrowed-looking figures; the systems detail is useful, but the headline claim needs major revision. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the decision layer and its trigger point $\Gamma$: a scalar threshold that the shallow part's softmax confidence $\beta$ is tested against, deciding whether inference stops or the deep part is activated. The decision layer also factors in the desired accuracy and a set of high-priority classes, so $\Gamma$ is not fixed but can be raised when, say, a high-priority label appears in the shallow top-$n$ outputs. On the hardware side, the enabler is partial reconfiguration: the three convolution blocks are separate FPGA bitstreams of the same size, and only the needed block is loaded, with batch processing making the reconfiguration cost negligible.
What would settle it
Force every test image whose shallow confidence lands just above the trigger threshold to also pass through the deep part; if the deep part corrects a substantial fraction of those images, the confidence metric is not a reliable gate and the reported computation savings overstate what accuracy can be maintained.
Extended reading notes
Core claim
On the paper's own terms, the central discovery is that a confidence value taken from a shallow CNN block can act as a nearly free, accurate gate for deciding whether deeper computation is needed. In AH-CNN, each image first passes through a shallow part; the decision layer compares the softmax confidence $\beta$ with a trigger point $\Gamma$, and only when $\beta \le \Gamma$ does the system load and run the next convolution block via partial reconfiguration. The trigger point is derived from the mean and standard deviation of confidence values on the training set and can be raised dynamically for high-priority classes. With this routing, the paper reports Top-1 validation accuracies of 85.4% on CIFAR-10, 55.4% on CIFAR-100, and 94.2% on SVHN, close to the 86.27%, 56.60%, and 94.62% of the full three-part model, while using only 69.8%, 71.8%, and 43.8% of its computation and reaching roughly 400 images per second on SVHN.
Load-bearing premise
The load-bearing premise is that the shallow network's confidence in its top prediction reliably indicates when the deeper network would actually fix a mistake, and that the trigger threshold chosen on validation data keeps working on new images.
Editorial extensions
If this is right
- On CIFAR-10, CIFAR-100, and SVHN, the confidence gate runs only 69.8%, 71.8%, and 43.8% of the deepest network's computation, respectively, while holding Top-1 accuracy within about one point of the full model.
- A resource-constrained FPGA can host a deep CNN by splitting it into partial-reconfiguration blocks, because batch processing spreads the reconfiguration time over many images.
- The runtime accuracy-versus-speed tradeoff is controllable through the trigger threshold, so the same implementation can be made more conservative for critical classes or higher expected accuracy.
- Because the confidence gate is cheaper to compute than entropy or learned feed-forward gates, the routing decision itself adds little overhead to the inference pipeline.
Reading between the lines
- A natural next step is per-class or per-image trigger thresholds; the paper's own high-priority-class adjustment hints that a single global $\Gamma$ is not optimal for every category.
- The magnitude of the savings is dataset-dependent: on harder distributions with many confusable classes, more inputs will fall below the threshold and the compute savings shrink, while on easier distributions they grow.
- Because the gate consumes only the shallow output and no deep-layer features, the same confidence check could be placed in front of any deep model, not just the quantized ResNet-18 reported here, provided the shallow confidence is calibrated.
- The routing decision could be combined with other dynamic-inference levers, such as input resolution or layer skipping, since the gate runs before the deep part and could inform more than one resource decision.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes AH-CNN, an adaptive early-exit convolutional neural network implemented on a Xilinx Zynq FPGA with dynamic partial reconfiguration. A shallow part computes a softmax confidence; a decision layer compares that confidence to a threshold Γ and either exits with the shallow result or reconfigures the FPGA to load a deeper part. Experiments on CIFAR-10, CIFAR-100, and SVHN report that the routed system performs only 69.8%, 71.8%, and 43.8% of the deepest-network computation while reaching Top-1 validation accuracies of 85.4%, 55.4%, and 94.2%, respectively, with throughput up to 408 images/s on SVHN. The paper also reports FPGA resource usage, partial-reconfiguration times, and a comparison against SkipNet and entropy-based early-exit selection.
Significance. If the reported numbers are correct, the paper makes a useful contribution toward efficient embedded inference by combining early-exit routing with FPGA partial reconfiguration; the measured resource utilization (Table I) and execution times (Table II) are concrete and potentially valuable to practitioners. The authors also point to a public code repository (https://github.com/mfarhadi/AHCNN). However, the evaluation as written contains an internal accuracy inconsistency that undermines the central 'maintain desired accuracy' claim, and the threshold-based efficiency claim is not reproducible without reporting the threshold values and the validation protocol. The significance of the approach can only be assessed after the evaluation is corrected and made sufficiently detailed.
major comments (3)
- [Section VI-B, Table III] The reported AH-CNN Top-1 validation accuracy on SVHN (94.2%) is inconsistent with the deepest HLS-optimized IP-core in Table III, which reports SVHN Top-1 accuracy of 83.46% and Top-5 accuracy of 94.62%. In an early-exit system with no ensembling, oracle routing, or separate full-precision model, the overall Top-1 accuracy cannot exceed the Top-1 accuracy of the deepest part that is actually used; since only 43.8% of SVHN images are routed to Part 3 and the rest exit at Part 1 (72.14% Top-1), the system's Top-1 accuracy is bounded above by 83.46%. The text describes no mechanism that would explain the 10.7-point gap. Please provide a corrected per-branch accuracy table and clarify whether the 94.2% value is a typo or actually refers to Top-5 accuracy.
- [Sections V-A, V-B and Algorithm 1] The trigger threshold Γ is the central control parameter of the efficiency claim, but the paper never reports its value or the procedure that maps the desired accuracy Λ to a specific Γ. Section V-B states that the mean and range of the trigger point value 'are needed to be learned from the training data,' and Algorithm 1 says 'Assign proper Γ based on Λ' without specifying the assignment. The reported computation savings (69.8%, 71.8%, 43.8%) and the corresponding accuracies therefore depend on a threshold that appears to be selected in-sample; without reporting Γ and Θ for each dataset and showing the full accuracy-versus-computation tradeoff curve on a held-out validation set, the efficiency claim is not reproducible and may reflect threshold tuning rather than a genuine property of the adaptive system.
- [Section VI-B, Figure 5] The comparison with SkipNet and entropy-based selection reports only FLOPs, not the classification accuracy achieved by each method at the reported operating points. Because the paper's central claim is that the confidence-based method 'can maintain the desired accuracy' while reducing computation, a computation-only comparison does not establish that the confidence method is superior in the accuracy/efficiency tradeoff. Please provide a table or plot showing accuracy and computation for all compared methods at the same operating points (or, if accuracy is identical by construction, say so explicitly).
minor comments (5)
- [Table III] The header of Table III is ambiguous: 'CIFAR10 CIFAR100 SVHNTop1 Top5' can be read as three or four columns, and the Top-5 accuracy for CIFAR-10 and CIFAR-100 is not listed; please reformat the table with clear column titles and consistent metrics.
- [Figure 5 caption] The caption states that confidence reduces computation by '≈30%, ≈27% and ≈57%' on CIFAR-10, CIFAR-100, and SVHN, while the text reports computation ratios of 69.8%, 71.8%, and 43.8% (i.e., reductions of 30.2%, 28.2%, and 56.2%); the 27% versus 28.2% discrepancy should be corrected.
- [Section VI-B] The throughput numbers (268, 217, 408 images/s) should be reconciled with the statement that applying all parts to images gives ≈160 images/s; for SVHN, 408/160 = 2.55× while the computation ratio 43.8% implies an expected speedup of 2.28×, so please clarify whether partial-reconfiguration overhead and batch size are included in both numbers.
- [Section V-B, loss function] The joint loss function is mathematically malformed: L(ŷ,y;θ) = Σ_N L(ŷ_n,y;θ) is followed by a definition that uses ζ both as the set of labels and as a denominator, and the indices on y, x, and k are inconsistent; please rewrite this equation with clear notation for the number of classification modules, the sum over labels, and the sum over training samples.
- [Section I, Figure 1] Figure 1 appears to be a reproduction of a figure from reference [8] with its original caption; please confirm that permission has been obtained or remove the figure, since it is not essential to the FPGA/CNN contribution.
Circularity Check
The headline computation-savings percentages are the direct routing outcome of the confidence threshold fitted on training data, making the efficiency claim an operating-point report rather than an independent result; the SVHN accuracy-maintenance claim is additionally internally inconsistent.
-
fitted input called prediction
[Section V (Training Phase), implemented in Algorithm 1 (Section III-A); reported in Section VI-B]
"The mean and range of trigger point value are needed to be learned from the training data."
Reported savings are the direct routing outcome of the fitted trigger point. Section V says the trigger point's 'mean and range ... needed to be learned from the training data'; Algorithm 1 routes to the deep part 'if β <= Γ' and first 'Assign proper Γ based on Λ'. So the abstract's 'only 69.8%, 71.8%, and 43.8% of the computation in the deepest network is done' is the empirical fraction of validation images falling on the deep side of the chosen Γ, not an independent prediction; changing Γ changes the percentages by construction. Accuracy-maintenance, which could give independent content, is unsupported as reported: Sec. VI-B gives SVHN Top-1 94.2% for AH-CNN while Table III gives deepest Part 3 83.46% Top-1 on SVHN, with no mechanism described.
full rationale
The paper's FPGA contribution is genuinely self-contained: partial reconfiguration, HLS IP cores, and comparisons to SkipNet and entropy selection are external benchmarks that do not depend on self-citation. However, the central efficiency claim reduces to threshold selection: Γ is fitted from training data, and the reported computation percentages (69.8%, 71.8%, 43.8%) are simply the routing fractions induced by that fitted threshold. The one component that could make the result non-circular—maintaining the desired accuracy on the routed subset—is not established as reported, because the SVHN routed Top-1 of 94.2% exceeds the deepest HLS IP-core's 83.46% with no described mechanism. Hence a partial circularity score of 6 is appropriate: the headline savings are fitted operating-point outcomes, while the accuracy-maintenance claim is unverified and internally inconsistent.
Assumptions & free parameters
free parameters (3)
- Confidence threshold Γ (trigger point) =
not reported (derived from mean and standard deviation of confidence values over the training set)
- Priority increment Θ =
not reported
- Quantization widths =
1-bit weights, 5-bit activations
assumptions (4)
- domain assumption Softmax confidence of the shallow part is a valid indicator of whether the deep part will improve classification.
- domain assumption The human visual system analogy justifies a two-stage early-exit design.
- domain assumption Batch processing makes partial reconfiguration overhead negligible.
- standard math Standard stochastic gradient training with cross-entropy loss minimizes the joint loss correctly.
Cite this review
Pith. "Pith review of A Novel Design of Adaptive and Hierarchical Convolutional Neural Networks using Partial Reconfiguration on FPGA." pith.science (2026). https://pith.science/paper/465LZFZC
@misc{pith2026190905653,
author = {Pith},
title = {Pith review of: A Novel Design of Adaptive and Hierarchical Convolutional Neural Networks using Partial Reconfiguration on FPGA},
year = {2026},
howpublished = {\url{https://pith.science/paper/465LZFZC}},
note = {Machine review of arXiv:1909.05653}
}
read the original abstract
Nowadays most research in visual recognition using Convolutional Neural Networks (CNNs) follows the "deeper model with deeper confidence" belief to gain a higher recognition accuracy. At the same time, deeper model brings heavier computation. On the other hand, for a large chunk of recognition challenges, a system can classify images correctly using simple models or so-called shallow networks. Moreover, the implementation of CNNs faces with the size, weight, and energy constraints on the embedded devices. In this paper, we implement the adaptive switching between shallow and deep networks to reach the highest throughput on a resource-constrained MPSoC with CPU and FPGA. To this end, we develop and present a novel architecture for the CNNs where a gate makes the decision whether using the deeper model is beneficial or not. Due to resource limitation on FPGA, the idea of partial reconfiguration has been used to accommodate deep CNNs on the FPGA resources. We report experimental results on CIFAR-10, CIFAR-100, and SVHN datasets to validate our approach. Using confidence metric as the decision making factor, only 69.8%, 71.8%, and 43.8% of the computation in the deepest network is done for CIFAR-10, CIFAR-100, and SVHN while it can maintain the desired accuracy with the throughput of around 400 images per second for SVHN dataset.
Figures
Reference graph
Works this paper leans on
-
[8]
R. M. Cichy, D. Pantazis, and A. Oliva. Similarity-based fusion of meg and fmri reveals spatio-temporal dynamics in human cortex during visual object recognition. Cerebral Cortex, 26(8):3563–3579, 2016
work page 2016
-
[1]
M. Al Kadi, P. Rudolph, D. Gohringer, and M. Hubner. Dynamic and partial reconfiguration of zynq 7000 under linux. In 2013 International Conference on Reconfigurable Computing and FPGAs (ReConFig) , pages 1–5. IEEE, 2013
work page 2013
- [2]
-
[3]
E. Bengio, P.-L. Bacon, J. Pineau, and D. Precup. Conditional computation in neural networks for faster models. arXiv preprint arXiv:1511.06297, 2015
arXiv 2015
-
[4]
T. Bolukbasi, J. Wang, O. Dekel, and V . Saligrama. Adaptive neural networks for efficient inference. In International Conference on Machine Learning, pages 527–536, 2017
work page 2017
-
[5]
C. Chen, A. Seff, A. Kornhauser, and J. Xiao. Deepdriving: Learning affordance for direct perception in autonomous driving. In Proceedings of the IEEE International Conference on Computer Vision , pages 2722– 2730, 2015
work page 2015
-
[6]
T. Y .-h. Chen. Glimpse : Continuous , Real-Time Object Recognition on Mobile Devices Categories and Subject Descriptors. SenSys ’15 Proceedings of the 13th ACM Conference on Embedded Networked Sensor Systems, pages 155–168, 2015
work page 2015
-
[7]
R. M. Cichy, D. Pantazis, and A. Oliva. Resolving human object recognition in space and time. Nature Neuroscience , 17(3):455–462, 2014
work page 2014
Show all 43 references
-
[9]
J. Dean, G. Corrado, R. Monga, K. Chen, M. Devin, M. Mao, A. Senior, P. Tucker, K. Yang, Q. V . Le, et al. Large scale distributed deep networks. In Advances in neural information processing systems , pages 1223– 1231, 2012
2012
-
[10]
T. Feist. Vivado design suite. 2012
2012
-
[11]
Glorot and Y
X. Glorot and Y . Bengio. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics , pages 249–256, 2010
2010
-
[12]
Y . Gong, L. Liu, M. Yang, and L. Bourdev. Compressing deep convolutional networks using vector quantization. arXiv preprint arXiv:1412.6115, 2014
2014 arXiv
-
[13]
S. Han, H. Mao, and W. J. Dally. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv preprint arXiv:1510.00149 , 2015
2015 arXiv
-
[14]
S. Han, H. Mao, and W. J. Dally. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. In Proceedings of the International Conference on Learning Representations, 2016
2016
-
[15]
S. Han, J. Pool, J. Tran, and W. Dally. Learning both weights and connections for efficient neural network. In Advances in Neural Information Processing Systems , pages 1135–1143, 2015
2015
-
[16]
K. He, X. Zhang, S. Ren, and J. Sun. Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778, 2016
2016
-
[17]
K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 770–778, 2016
2016
-
[18]
F. N. Iandola, S. Han, M. W. Moskewicz, K. Ashraf, W. J. Dally, and K. Keutzer. Squeezenet: Alexnet-level accuracy with 50x fewer parameters and¡ 0.5 mb model size. arXiv preprint arXiv:1602.07360 , 2016
2016 arXiv
-
[19]
Izadyyazdanabadi, E
M. Izadyyazdanabadi, E. Belykh, M. Mooney, N. Martirosyan, J. Es- chbacher, P. Nakaji, M. C. Preul, and Y . Yang. Convolutional neural networks: Ensemble modeling, fine-tuning and unsupervised semantic localization. arXiv preprint arXiv:1709.03028 , 2017
2017 arXiv
-
[20]
Jokic, S
P. Jokic, S. Emery, and L. Benini. Binaryeye: A 20 kfps streaming camera system on fpga with real-time on-device image recognition using binary neural networks. In 2018 IEEE 13th International Symposium on Industrial Embedded Systems (SIES) , pages 1–7. IEEE, 2018
2018
-
[21]
K ¨astner, B
F. K ¨astner, B. Janßen, F. Kautz, M. H ¨ubner, and G. Corradi. Hard- ware/software codesign for convolutional neural networks exploiting dynamic partial reconfiguration on pynq. In 2018 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW) , pages...
2018
-
[22]
Krizhevsky
A. Krizhevsky. Learning multiple layers of features from tiny images. 2009
2009
-
[23]
Krizhevsky, I
A. Krizhevsky, I. Sutskever, and G. E. Hinton. ImageNet Classifica- tion with Deep Convolutional Neural Networks. Advances In Neural Information Processing Systems , pages 1–9, 2012
2012
-
[24]
C. Lea, R. Vidal, and G. D. Hager. Learning convolutional action primitives for fine-grained action recognition. In Robotics and Automa- tion (ICRA), 2016 IEEE International Conference on , pages 1642–1649. IEEE, 2016
2016
-
[25]
H. Li, Z. Lin, X. Shen, J. Brandt, and G. Hua. A convolutional neural network cascade for face detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 5325– 5334, 2015
2015
-
[26]
Netzer, T
Y . Netzer, T. Wang, A. Coates, A. Bissacco, B. Wu, and A. Y . Ng. Reading digits in natural images with unsupervised feature learning. 2011
2011
-
[27]
J. Qiu, J. Wang, S. Yao, K. Guo, B. Li, E. Zhou, J. Yu, T. Tang, N. Xu, S. Song, et al. Going deeper with embedded fpga platform for convolutional neural network. In Proceedings of the 2016 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays , pages 26–35. ACM, 2016
2016
-
[28]
Rastegari, V
M. Rastegari, V . Ordonez, J. Redmon, and A. Farhadi. Xnor-net: Imagenet classification using binary convolutional neural networks. In European Conference on Computer Vision , pages 525–542. Springer, 2016
2016
-
[29]
Rav `ı, C
D. Rav `ı, C. Wong, F. Deligianni, M. Berthelot, J. Andreu-Perez, B. Lo, and G.-Z. Yang. Deep learning for health informatics. IEEE journal of biomedical and health informatics , 21(1):4–21, 2017
2017
-
[30]
Redmon and A
J. Redmon and A. Farhadi. Yolo9000: better, faster, stronger. arXiv preprint arXiv:1612.08242, 2016
2016 arXiv
-
[31]
J. B. Ritchie, D. A. Tovar, and T. A. Carlson. Emerging Object Represen- tations in the Visual System Predict Reaction Times for Categorization. PLoS Computational Biology , 11(6):1–18, 2015
2015
-
[32]
H. Shen, S. Han, M. Philipose, and A. Krishnamurthy. Fast video classification via adaptive cascading of deep models. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2017
2017
-
[33]
Szegedy, W
C. Szegedy, W. Liu, Y . Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V . Vanhoucke, and A. Rabinovich. Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1–9, 2015
2015
-
[34]
J. Tang, D. Sun, S. Liu, and J.-L. Gaudiot. Enabling deep learning on iot devices. Computer, 50(10):92–96, 2017
2017
-
[35]
Teerapittayanon, B
S. Teerapittayanon, B. McDanel, and H. Kung. Branchynet: Fast inference via early exiting from deep neural networks. In Pattern Recognition (ICPR), 2016 23rd International Conference on , pages 2464–2469. IEEE, 2016
2016
-
[36]
Umuroglu, N
Y . Umuroglu, N. J. Fraser, G. Gambardella, M. Blott, P. Leong, M. Jahre, and K. Vissers. Finn: A framework for fast, scalable binarized neural network inference. In Proceedings of the 2017 ACM/SIGDA Interna- tional Symposium on Field-Programmable Gate Arrays , pages 65–74. ACM, 2017
2017
-
[37]
Viola and M
P. Viola and M. Jones. Rapid object detection using a boosted cascade of simple features. In Computer Vision and Pattern Recognition,
-
[38]
X. Wang, F. Yu, Z.-Y . Dou, T. Darrell, and J. E. Gonzalez. Skipnet: Learning dynamic routing in convolutional networks. In Proceedings of the European Conference on Computer Vision (ECCV) , pages 409–424, 2018
2018
-
[39]
W. Wen, C. Wu, Y . Wang, Y . Chen, and H. Li. Learning structured sparsity in deep neural networks. In Advances in Neural Information Processing Systems, pages 2074–2082, 2016
2016
-
[40]
M. D. Zeiler and R. Fergus. Visualizing and understanding convolutional networks. Lecture Notes in Computer Science (including subseries Lec- ture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 8689 LNCS(PART 1):818–833, 2014
2014
-
[41]
B. Zhou, A. Lapedriza, J. Xiao, A. Torralba, and A. Oliva. Learning deep features for scene recognition using places database. In Advances in neural information processing systems , pages 487–495, 2014
2014
-
[42]
Zhou, B.-B
H.-Y . Zhou, B.-B. Gao, and J. Wu. Adaptive feeding: Achieving fast and accurate detections by adaptively combining object detectors. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 3505–3513, 2017
2017
-
[2001]
Proceedings of the 2001 IEEE Computer Society Conference on, volume 1, pages I–I
CVPR 2001. Proceedings of the 2001 IEEE Computer Society Conference on, volume 1, pages I–I. IEEE, 2001
2001
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.