{"id":"ca0e0448-cb3f-4546-af91-3aaee0b5053b","arxiv_id":"1909.05653","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"An FPGA implementation of adaptive early-exit CNNs with partial reconfiguration saves computation on easy images, but the reported accuracy savings are not self-consistent.","lead":"This paper builds an FPGA system that runs a small neural network first and only switches to a bigger network when the small one is unsure, using partial reconfiguration to fit the network on chip. The authors report cutting computation to as low as 44% of the deepest model on three image benchmarks, but the accuracy numbers are internally inconsistent.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported SVHN accuracy for the routed AH-CNN (94.2% Top-1) is inconsistent with the deepest FPGA part (83.46% Top-1), so the accuracy-maintenance part of the central claim is not supported as reported.","rationale":"I focus on the accuracy-maintenance half of the central claim because the computation-reduction numbers can only be interpreted relative to a preserved accuracy. The text supplies exactly one accuracy claim for the full AH-CNN (94.2% SVHN Top-1) and exactly one accuracy for the deepest quantized part (83.46% SVHN Top-1). Under the natural reading these conflict. If 94.2% refers to a full-precision PyTorch model rather than the quantized FPGA IP cores, the paper never says so, and then it is not the accuracy of the system whose computation is being counted. If 94.2% is a top-5 figure, it is mislabeled. Either way the reader cannot check the claim that the FPGA system maintains accuracy at 43.8% computation. The reported threshold fitting in Section V and Algorithm 1 is another soft spot, but it is secondary to this inconsistency: even a perfectly calibrated confidence gate cannot justify an accuracy number that contradicts the component accuracies without an explicit explanation. I credit the authors for releasing code and for a plausible partial-reconfiguration framework; the objection is to the evidence, not the concept. A revised version that reports a standard accuracy-vs-computation table (with test-set accuracy and computation fraction for each Γ) for the actual quantized FPGA implementation could resolve this.","tokens_in":10792,"tokens_out":8536,"duration_ms":83071,"concrete_test":"Run the released implementation (github.com/mfarhadi/AHCNN) on SVHN: (1) measure standalone Part 3 Top-1 accuracy on the validation/test split used for Table III; (2) simulate Algorithm 1 over a sweep of Γ and record both end-to-end Top-1 accuracy and the fraction of deep-part computation; (3) check whether any Γ simultaneously yields roughly 43.8% deep computation and accuracy at least as high as Part 3 alone. Also check whether 94.2% is reproducible from the quantized model; if not, the discrepancy is a typo or data mismatch and the central claim lacks support.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section VI-B reports 'AH-CNN model achieves 85.4%, 55.4%, 94.2% Top-1 validation accuracy over CIFAR10, CIFAR-100, and SVHN respectively.' The same section's Table III reports the accuracy of the HLS-optimized IP-cores: the deepest part (Part 3) has SVHN Top-1 accuracy 83.46%, with SVHN Top-5 94.62%. On its face, the routed system's SVHN Top-1 exceeds the deepest part's Top-1 by 10.7 points. The paper describes no mechanism (oracle routing, ensembling, or a separate full-precision model) that would explain this. A confidence-gated early-exit system can in principle beat the deep network on easy subsets, but the reported gap would require the shallow head to be far more accurate than the deep network on the high-confidence subset while the deep part is run on only 43.8% of images; this is so surprising that the numbers cannot be accepted without a per-dataset accuracy/computation table for the exact quantized routed model. The core efficiency claim ('maintain desired accuracy') is therefore unverifiable from the text as written.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes AH-CNN, an adaptive early-exit convolutional neural network implemented on a Xilinx Zynq FPGA with dynamic partial reconfiguration. A shallow part computes a softmax confidence; a decision layer compares that confidence to a threshold Γ and either exits with the shallow result or reconfigures the FPGA to load a deeper part. Experiments on CIFAR-10, CIFAR-100, and SVHN report that the routed system performs only 69.8%, 71.8%, and 43.8% of the deepest-network computation while reaching Top-1 validation accuracies of 85.4%, 55.4%, and 94.2%, respectively, with throughput up to 408 images/s on SVHN. The paper also reports FPGA resource usage, partial-reconfiguration times, and a comparison against SkipNet and entropy-based early-exit selection.","tokens_in":11028,"tokens_out":6030,"duration_ms":57683,"significance":"If the reported numbers are correct, the paper makes a useful contribution toward efficient embedded inference by combining early-exit routing with FPGA partial reconfiguration; the measured resource utilization (Table I) and execution times (Table II) are concrete and potentially valuable to practitioners. The authors also point to a public code repository (https://github.com/mfarhadi/AHCNN). However, the evaluation as written contains an internal accuracy inconsistency that undermines the central 'maintain desired accuracy' claim, and the threshold-based efficiency claim is not reproducible without reporting the threshold values and the validation protocol. The significance of the approach can only be assessed after the evaluation is corrected and made sufficiently detailed.","major_comments":[{"comment":"The reported AH-CNN Top-1 validation accuracy on SVHN (94.2%) is inconsistent with the deepest HLS-optimized IP-core in Table III, which reports SVHN Top-1 accuracy of 83.46% and Top-5 accuracy of 94.62%. In an early-exit system with no ensembling, oracle routing, or separate full-precision model, the overall Top-1 accuracy cannot exceed the Top-1 accuracy of the deepest part that is actually used; since only 43.8% of SVHN images are routed to Part 3 and the rest exit at Part 1 (72.14% Top-1), the system's Top-1 accuracy is bounded above by 83.46%. The text describes no mechanism that would explain the 10.7-point gap. Please provide a corrected per-branch accuracy table and clarify whether the 94.2% value is a typo or actually refers to Top-5 accuracy.","section":"Section VI-B, Table III"},{"comment":"The trigger threshold Γ is the central control parameter of the efficiency claim, but the paper never reports its value or the procedure that maps the desired accuracy Λ to a specific Γ. Section V-B states that the mean and range of the trigger point value 'are needed to be learned from the training data,' and Algorithm 1 says 'Assign proper Γ based on Λ' without specifying the assignment. The reported computation savings (69.8%, 71.8%, 43.8%) and the corresponding accuracies therefore depend on a threshold that appears to be selected in-sample; without reporting Γ and Θ for each dataset and showing the full accuracy-versus-computation tradeoff curve on a held-out validation set, the efficiency claim is not reproducible and may reflect threshold tuning rather than a genuine property of the adaptive system.","section":"Sections V-A, V-B and Algorithm 1"},{"comment":"The comparison with SkipNet and entropy-based selection reports only FLOPs, not the classification accuracy achieved by each method at the reported operating points. Because the paper's central claim is that the confidence-based method 'can maintain the desired accuracy' while reducing computation, a computation-only comparison does not establish that the confidence method is superior in the accuracy/efficiency tradeoff. Please provide a table or plot showing accuracy and computation for all compared methods at the same operating points (or, if accuracy is identical by construction, say so explicitly).","section":"Section VI-B, Figure 5"}],"minor_comments":[{"comment":"The header of Table III is ambiguous: 'CIFAR10 CIFAR100 SVHNTop1 Top5' can be read as three or four columns, and the Top-5 accuracy for CIFAR-10 and CIFAR-100 is not listed; please reformat the table with clear column titles and consistent metrics.","section":"Table III"},{"comment":"The caption states that confidence reduces computation by '≈30%, ≈27% and ≈57%' on CIFAR-10, CIFAR-100, and SVHN, while the text reports computation ratios of 69.8%, 71.8%, and 43.8% (i.e., reductions of 30.2%, 28.2%, and 56.2%); the 27% versus 28.2% discrepancy should be corrected.","section":"Figure 5 caption"},{"comment":"The throughput numbers (268, 217, 408 images/s) should be reconciled with the statement that applying all parts to images gives ≈160 images/s; for SVHN, 408/160 = 2.55× while the computation ratio 43.8% implies an expected speedup of 2.28×, so please clarify whether partial-reconfiguration overhead and batch size are included in both numbers.","section":"Section VI-B"},{"comment":"The joint loss function is mathematically malformed: L(ŷ,y;θ) = Σ_N L(ŷ_n,y;θ) is followed by a definition that uses ζ both as the set of labels and as a denominator, and the indices on y, x, and k are inconsistent; please rewrite this equation with clear notation for the number of classification modules, the sum over labels, and the sum over training samples.","section":"Section V-B, loss function"},{"comment":"Figure 1 appears to be a reproduction of a figure from reference [8] with its original caption; please confirm that permission has been obtained or remove the figure, since it is not essential to the FPGA/CNN contribution.","section":"Section I, Figure 1"}],"recommendation":"major_revision","confidential_remarks":"The core idea is plausible and the FPGA implementation details are a worthwhile contribution, but the inconsistent SVHN accuracy numbers (94.2% overall vs. 83.46% for the deepest part) are a serious correctness issue that must be resolved with corrected tables and, if necessary, rerun experiments. The underspecified threshold selection also prevents reproducibility. I recommend major revision, but if the authors cannot produce consistent accuracy/computation numbers, the paper should be rejected."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick take: there is a real systems effort here – an early-exit ResNet-18 on a Pynq-Z1 using dynamic partial reconfiguration, with measured reconfiguration times and shipped code – but the paper does not substantiate its headline accuracy-preserving efficiency claim. The SVHN numbers alone should block acceptance: Table III gives the deepest FPGA part 83.46% top-1 on SVHN, while Section VI-B says the routed AH-CNN gets 94.2% top-1. A confidence-gated router can in principle beat a deep-only model on some subsets, but with the deep part used on only 43.8% of images you need a per-subset accuracy table to make that credible; without one, this reads as an arithmetic error, and it sits at the center of the central claim.\n\nWhat is genuinely useful is the implementation detail: three partial bitstreams of 2.4 MB each, 38–42 ms reconfiguration time, 2 ms FPGA execution per part, about 400 images/s on SVHN, and a resource table showing why partial reconfiguration is needed. The comparison of confidence versus entropy versus SkipNet-style gates is the right question, and the code is on GitHub. That is worth preserving.\n\nThe soft spots beyond the numbers: the conceptual novelty is thin. BranchyNet already did confidence/entropy early exit, and Kästner et al. already demonstrated partial reconfiguration for CNNs on Pynq. The new bit is a priority-class threshold bump and a concrete combination, which is engineering progress, not a new method. The threshold Γ is learned from training data and the savings are reported on the validation set where it was tuned, so the 69.8/71.8/43.8% figures should be presented as an achieved operating point, not as a predictive result. Table III is also malformed, and Figures 1 and 5 look reused without clear credit: Figure 1 is a neuroscience figure from [8], and Figure 5 is written entirely in SkipNet's notation (SP_G1, SP_G2, RL_G1, RL_G2) with no source acknowledgment.\n\nThis is for people working on embedded FPGA inference. A serious referee could help fix the evaluation and separate the real hardware contribution from the overclaimed novelty. As submitted, I would not accept it, but I would send it to review if the authors add a proper accuracy/efficiency sweep, resolve the SVHN inconsistency, and clean up the figure provenance.","headline":"Real FPGA partial-reconfiguration work undercut by an internal SVHN accuracy contradiction and borrowed-looking figures; the systems detail is useful, but the headline claim needs major revision.","tokens_in":11587,"tokens_out":6045,"would_cite":false,"duration_ms":62842,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"An adaptive CNN with a confidence-based gate preserves deep-model accuracy on embedded hardware while running as little as 44% of the computation.","keywords":["adaptive neural networks","early exit","confidence-based gating","FPGA","partial reconfiguration","quantized CNNs","embedded inference","image classification"],"falsifier":"Force every test image whose shallow confidence lands just above the trigger threshold to also pass through the deep part; if the deep part corrects a substantial fraction of those images, the confidence metric is not a reliable gate and the reported computation savings overstate what accuracy can be maintained.","tokens_in":10565,"feed_emoji":"⚡","tokens_out":10791,"duration_ms":105891,"temperature":0.7,"pith_summary":"This paper proposes an adaptive hierarchical CNN, AH-CNN, that answers easy images with a shallow network and sends only hard images to deeper layers, using the shallow network's confidence as the gate. The payoff is that deep-model accuracy can be kept on a resource-limited CPU+FPGA platform while executing only 69.8%, 71.8%, and 43.8% of the deepest network's computation on CIFAR-10, CIFAR-100, and SVHN, with throughput around 400 images per second on SVHN. To fit the full model on the FPGA, the network is split into parts and partial reconfiguration swaps deeper convolution blocks in only when the gate calls for them. The experimental comparison shows the confidence signal is cheaper than entropy-based or learned-gate alternatives while making the same routing decisions.","feed_headline":"Confidence gate cuts deep-CNN compute by up to 56%","feed_subtitle":"Easy images exit after a shallow net; the FPGA loads deeper layers only for hard ones, and Top-1 accuracy holds.","key_machinery":"The load-bearing mechanism is the decision layer and its trigger point $\\Gamma$: a scalar threshold that the shallow part's softmax confidence $\\beta$ is tested against, deciding whether inference stops or the deep part is activated. The decision layer also factors in the desired accuracy and a set of high-priority classes, so $\\Gamma$ is not fixed but can be raised when, say, a high-priority label appears in the shallow top-$n$ outputs. On the hardware side, the enabler is partial reconfiguration: the three convolution blocks are separate FPGA bitstreams of the same size, and only the needed block is loaded, with batch processing making the reconfiguration cost negligible.","core_discovery":"On the paper's own terms, the central discovery is that a confidence value taken from a shallow CNN block can act as a nearly free, accurate gate for deciding whether deeper computation is needed. In AH-CNN, each image first passes through a shallow part; the decision layer compares the softmax confidence $\\beta$ with a trigger point $\\Gamma$, and only when $\\beta \\le \\Gamma$ does the system load and run the next convolution block via partial reconfiguration. The trigger point is derived from the mean and standard deviation of confidence values on the training set and can be raised dynamically for high-priority classes. With this routing, the paper reports Top-1 validation accuracies of 85.4% on CIFAR-10, 55.4% on CIFAR-100, and 94.2% on SVHN, close to the 86.27%, 56.60%, and 94.62% of the full three-part model, while using only 69.8%, 71.8%, and 43.8% of its computation and reaching roughly 400 images per second on SVHN.","pith_inferences":["A natural next step is per-class or per-image trigger thresholds; the paper's own high-priority-class adjustment hints that a single global $\\Gamma$ is not optimal for every category.","The magnitude of the savings is dataset-dependent: on harder distributions with many confusable classes, more inputs will fall below the threshold and the compute savings shrink, while on easier distributions they grow.","Because the gate consumes only the shallow output and no deep-layer features, the same confidence check could be placed in front of any deep model, not just the quantized ResNet-18 reported here, provided the shallow confidence is calibrated.","The routing decision could be combined with other dynamic-inference levers, such as input resolution or layer skipping, since the gate runs before the deep part and could inform more than one resource decision."],"forward_implications":["On CIFAR-10, CIFAR-100, and SVHN, the confidence gate runs only 69.8%, 71.8%, and 43.8% of the deepest network's computation, respectively, while holding Top-1 accuracy within about one point of the full model.","A resource-constrained FPGA can host a deep CNN by splitting it into partial-reconfiguration blocks, because batch processing spreads the reconfiguration time over many images.","The runtime accuracy-versus-speed tradeoff is controllable through the trigger threshold, so the same implementation can be made more conservative for critical classes or higher expected accuracy.","Because the confidence gate is cheaper to compute than entropy or learned feed-forward gates, the routing decision itself adds little overhead to the inference pipeline."],"supporting_citations":[{"why":"Supplies the early-exit baseline that uses entropy of branch outputs for routing decisions.","marker":"[35]"},{"why":"Provides the SkipNet learned-gate baseline whose high gate cost is the reason confidence-based routing is preferred here.","marker":"[38]"},{"why":"Gives the entropy-based stacked-model selection method used as a second comparison baseline.","marker":"[4]"},{"why":"Provides the binarized neural-network inference approach that motivates the 1-bit weight, 5-bit activation quantized design.","marker":"[36]"},{"why":"Prior demonstration that per-layer dynamic reconfiguration of CNNs is expensive, motivating the block-level partial reconfiguration used here.","marker":"[21]"},{"why":"Defines the ResNet-18 architecture that is split into the shallow and deep parts of AH-CNN.","marker":"[17]"},{"why":"Supports the claim that early layers capture low-level features while later layers extract high-level features, which justifies early exits.","marker":"[40]"},{"why":"Supplies the CIFAR-10 and CIFAR-100 datasets used in the accuracy and compute-reduction experiments.","marker":"[22]"},{"why":"Supplies the SVHN dataset used for the throughput and compute-reduction experiments.","marker":"[26]"}],"fun_headline_variants":["FPGA reconfigures CNN depth on the fly based on confidence","Confidence gate triggers deeper CNN only when needed","Shallow net handles easy images; FPGA swaps in deep net for hard ones","Partial reconfiguration enables adaptive CNN depth on FPGA","Adaptive CNN cuts compute up to 56% while holding accuracy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the shallow network's confidence in its top prediction reliably indicates when the deeper network would actually fix a mistake, and that the trigger threshold chosen on validation data keeps working on new images.","fun_headline_variants_meta":{"raw":{"variants":["FPGA reconfigures CNN depth on the fly based on confidence","Confidence gate triggers deeper CNN only when needed","Shallow net handles easy images; FPGA swaps in deep net for hard ones","Partial reconfiguration enables adaptive CNN depth on FPGA","Adaptive CNN cuts compute up to 56% while holding accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000408,"raw_usage":{"total_tokens":2165,"prompt_tokens":1037,"completion_tokens":1128,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":653,"completion_tokens_details":{"reasoning_tokens":1043}},"tokens_in":653,"tokens_out":1128,"duration_ms":10580,"temperature":1.0,"reasoning_tokens":1043,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T04:48:49.191630+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Force every test image whose shallow confidence lands just above the trigger threshold to also pass through the deep part; if the deep part corrects a substantial fraction of those images, the confidence metric is not a reliable gate and the reported computation savings overstate what accuracy can be maintained.","supporting_citations":[{"cited_title":"Teerapittayanon, B","cited_arxiv_id":null,"evidence_quote":"Supplies the early-exit baseline that uses entropy of branch outputs for routing decisions."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the SkipNet learned-gate baseline whose high gate cost is the reason confidence-based routing is preferred here."},{"cited_title":"Bolukbasi, J","cited_arxiv_id":null,"evidence_quote":"Gives the entropy-based stacked-model selection method used as a second comparison baseline."},{"cited_title":"Umuroglu, N","cited_arxiv_id":null,"evidence_quote":"Provides the binarized neural-network inference approach that motivates the 1-bit weight, 5-bit activation quantized design."},{"cited_title":"K ¨astner, B","cited_arxiv_id":null,"evidence_quote":"Prior demonstration that per-layer dynamic reconfiguration of CNNs is expensive, motivating the block-level partial reconfiguration used here."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supports the claim that early layers capture low-level features while later layers extract high-level features, which justifies early exits."},{"cited_title":"Netzer, T","cited_arxiv_id":null,"evidence_quote":"Supplies the SVHN dataset used for the throughput and compute-reduction experiments."}],"review_version":1}