Pith. sign in

REVIEW 3 major objections 7 minor 50 references

Zeroize, recover, repeat: one-pass DNN tuning for hard latency targets

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · glm-5.2

2026-07-09 22:51 UTC pith:V7NOWECO

load-bearing objection Practical one-shot latency-constrained DNN optimization with a predictor-accuracy gap that undercuts the 'hard constraint' claim the 3 major comments →

arxiv 2607.06922 v1 pith:V7NOWECO submitted 2026-07-08 cs.LG cs.CV

Latency-Constrained DNN Architecture Learning for Edge Systems using Zerorized Batch Normalization

Shuo Huai , Di Liu , Hao Kong , Weichen Liu , Ravi Subramaniam , Christian Makaya , Qian Lin This is my paper
classification cs.LG cs.CV
keywords batch normalizationneural network pruninglatency predictionedge computingmodel scalingone-shot trainingchannel importancehardware-aware optimization
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that a neural network's architecture—not its trained weights—is what determines its efficiency, and that the right architecture for a given latency budget can be found during a single training run by repeatedly zeroizing and then allowing recovery of Batch Normalization (BN) scaling parameters. The BN parameter γ acts as a per-channel importance signal: channels with small γ are zeroized (effectively removed), but because the optimizer carries momentum from prior gradients, zeroized channels can spontaneously recover if they turn out to matter for accuracy. This zero-recover cycle, repeated throughout training, lets the network explore a wider architecture space than fixed-ratio pruning methods. A separate machine-learning-based latency predictor (a small backpropagation network trained on measured single-layer latencies for a specific device) translates the target latency into a compression ratio at each zeroizing step, so the training process steers directly toward the latency constraint rather than toward a proxy like FLOPs. For models that are already too fast (simple architectures on powerful devices), the same framework first uniformly scales the model up and then compacts it back down to the latency target, eliminating the expensive search for scaling factors. The entire pipeline—compression or scaling, latency guidance, and final trained weights—is produced in one training process with no pre-training and no re-training stage.

Core claim

The load-bearing mechanism is the Zero-Recovery cycle on BN parameters: zeroizing γ and β for unimportant channels during training, then letting optimizer momentum naturally restore some of them, produces a better compact architecture than methods that permanently remove channels or rank importance only once. Coupled with a hardware-specific latency predictor that converts a latency constraint into a compression ratio in milliseconds rather than epochs of on-device measurement, this yields a model that satisfies a hard latency target in a single training run. The paper demonstrates this on VGG-19, ResNet-50, and GoogLeNet across two NVIDIA edge devices, meeting a 34 ms constraint while匹配 or,

What carries the argument

Batch Normalization scaling parameter γ as a global channel-importance signal; Zero-Recovery training cycle (zeroize γ/β for unimportant channels, let momentum restore them); BP-neural-network latency predictor trained on measured single-layer latencies; unified scaling (uniformly expand width and depth, then compact back to latency target); layer pruning via added branch layers whose aggregate importance determines layer removal

Load-bearing premise

The latency predictor's average error of 6.12% (rising to 13.41% for some architectures like Inception) is small enough that the compression ratio it selects during training will produce a final model that actually satisfies the hard latency constraint when measured on the real device.

What would settle it

Measure the final latency of every model the framework produces on the target device; if a non-trivial fraction violate the 34 ms constraint, the one-shot claim fails. Separately, ablate the Recovery phase (zeroize only, never recover) and compare final accuracy—if recovery does not improve architecture quality, the central mechanism is decorative.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper proposes a framework for optimizing DNN architectures to meet hard latency constraints on edge devices. The framework has three components: (1) a compact learning scheme that dynamically zeroizes and recovers Batch Normalization (BN) parameters (γ, β) during a single training process, guided by global channel importance ranking; (2) a BP-neural-network-based latency predictor that estimates per-layer latency on a target device and aggregates along the topological critical path; (3) a model scaling scheme that uniformly expands simple models before compacting them, eliminating the need for expensive scaling-factor search. Experiments on CIFAR-10 and ImageNet-100 across VGG-19, ResNet, DenseNet, and GoogLeNet demonstrate higher accuracy than NS, SFP, FPGM, PGMPF, OTO, EfficientNet, and HACScale under a 34 ms latency constraint on NVIDIA Jetson TX2 and Nano, with minimal training overhead (1.12% over standard training). The code is open-sourced.

Significance. The paper addresses a practically important problem: deploying DNNs on edge devices under strict latency constraints. The Zero-Recovery mechanism (dynamically zeroizing and recovering BN parameters via momentum) is a clean and well-justified idea, and the mathematical derivation in Eq. (3)-(8) correctly shows how momentum enables recovery of zeroized parameters even when ReLU derivatives are zero. The one-shot training paradigm (avoiding pre-training and re-training) is a meaningful efficiency contribution, and the training overhead evaluation in Table 2 is a strong point. The open-source release at https://github.com/ntuliuteam/ZeroBN is commendable and supports reproducibility. The comprehensive comparison against seven state-of-the-art methods across four architectures and two devices is thorough.

major comments (3)
  1. §3.2 and Table 1: The BP-based latency predictor has an average error of 6.12%, with individual errors up to 13.41% (NasNet-mobile) and 8.31% (Inception-v3). Since GoogLeNet (used extensively in Table 4) is an Inception-based architecture, the predictor's reliability for this architecture class is a load-bearing concern for the 'hard' latency constraint claim. The paper does not report predicted-vs-actual latency at the selected compression ratio for any Table 4 entry, does not discuss whether errors are systematic (biased over- or under-estimation), and does not describe any safety margin applied when targeting 34 ms. While all Table 4 models land at 33.2±0.7 ms (all under 34 ms), the paper provides no mechanism ensuring this holds for untested architectures or devices. To support the one-shot hard-constraint claim (§4.4: 'our method only needs one time'), the authors should: (a) report
  2. §3.2: The input features to the BP network are not specified. The paper states the network has '16 neurons' in the input layer and takes 'a configuration' as input, but never lists what the 16 features are (e.g., kernel size, input/output channels, stride, layer type, etc.). This is a reproducibility gap that prevents others from constructing the predictor independently. While the open-source code may address this, the manuscript itself should specify the feature set, as the predictor is a core component of the framework.
  3. §4.4: The paper states that comparison methods need 'about 6-7 times' to meet the latency constraint via dichotomy search, while the proposed method needs 'one time.' It is unclear what 'times' refers to — is this 6-7 full training runs, or 6-7 on-device latency measurements? This distinction matters for the efficiency claim. Please clarify the comparison protocol and quantify the total wall-clock time difference, including the one-time cost of building the predictor's training dataset (device measurements of parameterized models, §3.2).
minor comments (7)
  1. §3.1.3: The equations between Eq. (7) and Eq. (8) are garbled/corrupted in the manuscript (the block containing /uni00000013/ etc.), making this section difficult to parse. Please fix the typesetting.
  2. §3.1.2, Line 14-16 of Algorithm 1: The layer importance threshold v_thre is mentioned but its value or method of determination is not specified in the experimental setup (§4.1). Please state how v_thre is set.
  3. §3.3: The scaling factor δ is described as set by 'the latency margin from the constraint' but no formula or procedure is given. Please specify how δ is computed.
  4. Table 4: The 'Comp. Ratio (%)' column contains negative values for scaling experiments (e.g., -37.51 for GoogLeNet unified scaling). It would help to clarify in the caption that negative ratios indicate model expansion rather than compression.
  5. §3.4, Fig. 10: The reference values for s and k are derived from experiments on VGG-19, ResNet-50, and GoogLeNet on ImageNet-100. It would strengthen the generality claim to note whether these reference values also hold for other datasets (e.g., CIFAR-10) or architectures (e.g., DenseNet).
  6. §2.3: The paper claims the BP predictor consumes 'much less time (a few hours)' compared to nn-Meter's 4.4 days, but does not break down this time into dataset construction (device measurement) vs. BP network training. Please provide this breakdown.
  7. The title uses 'Zerorized' which appears to be a non-standard term. If 'Zeroized' is intended (as used consistently in the body), please correct the title.

Circularity Check

0 steps flagged

No circularity found: the latency predictor is trained on independently sampled parameterized models, and the compression ratio is derived from predictor output plus training-determined channel importance, not from the target latency or final model.

full rationale

The paper's central claim is that its Zero-Recovery compact learning framework, guided by a BP-network latency predictor, can produce a DNN satisfying a hard latency constraint in a single training process. Walking the derivation chain: (1) The latency predictor (Section 3.2) is a BP neural network trained on parameterized single-layer models with various configurations sampled from the design space, with labels being measured on-device latencies. This training data is independent of the target model's final architecture or latency. (2) The compression ratio during Zero Training (Algorithm 1, Line 10) is computed as Predictor(l, Imp_rank, lp), where the channel importance ranking (Imp_rank) is determined by the trained γ values from the network's own training dynamics (Line 9), and l is the external latency constraint. The predictor maps this to a compression ratio; it does not assume the output latency equals the constraint. (3) The Zero-Recovery process (Sections 3.1.2–3.1.3) dynamically zeroizes and recovers BN parameters based on gradient-driven importance, with recovery enabled by momentum (Eqs. 6–8). This is a training procedure, not a definitional identity. (4) The model scaling scheme (Section 3.3) uniformly scales then compacts using the same independent learning process. No step reduces to its own inputs by construction. The self-citation [32] (Huai et al., DAC 2021) introduces the ZeroBN concept but is not load-bearing for the mathematical derivation — the equations and algorithm are self-contained in this paper. The skeptic's concern about predictor error (up to 13.41%) affecting the hard constraint guarantee is a correctness/robustness issue, not a circularity issue: the predictor is trained on independent data, and the final latencies in Table 4 are empirically measured, not defined to equal the constraint. The framework's claim is falsifiable by measuring actual latency of the output model, which the paper does. No prediction or first-principles result is equivalent to its inputs by construction.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 0 invented entities

The paper does not invent new mathematical entities or physical particles. It introduces an algorithmic framework (ZeroBN) and a predictor design, but these are methods, not axiomatic entities. The free parameters are standard hyperparameters of the training algorithm.

free parameters (5)
  • start epoch (s) = half of total epochs (heuristic)
    The epoch at which Zero training begins. The paper provides a heuristic (Section 3.4) but it is a hyperparameter that must be set per model/dataset.
  • zero interval (k)
    The epoch interval between Zero and Recovery training. Set to 2 in most instances (Section 3.4).
  • scaling factor (δ)
    The uniform scaling factor for model scaling. Set based on latency margin to ensure the scaled model exceeds the constraint (Section 3.3).
  • layer importance threshold (v_thre)
    Threshold for removing layers in layer pruning. Not explicitly specified how it is set.
  • BP network architecture = 3 layers (16-12-1 neurons)
    The architecture of the latency predictor is fixed but chosen empirically.
axioms (4)
  • domain assumption The γ parameter of a BN layer is a reliable proxy for channel importance.
    This is a standard assumption in network slimming [1], invoked in Section 3.1.1 to justify using γ for channel ranking.
  • domain assumption Momentum-based optimizers can recover zeroized parameters from past gradients.
    Invoked in Section 3.1.3 (Eq. 6-8) to justify the Recovery training phase. This is a property of standard SGD with momentum.
  • domain assumption Layer-wise latency prediction can be summed along the critical path to estimate model latency.
    Invoked in Section 3.2 to justify the latency predictor's topological analysis. This ignores potential overlapping or parallel execution on the hardware.
  • domain assumption Pre-training is unnecessary for finding efficient compact architectures.
    Invoked in Section 3.1, citing [14], to justify the one-shot training approach.

pith-pipeline@v1.1.0-glm · 28139 in / 2698 out tokens · 188664 ms · 2026-07-09T22:51:21.630807+00:00 · methodology

0 comments
read the original abstract

Deep learning applications have been widely adopted on edge devices, to mitigate the privacy and latency issues of accessing cloud servers. Deciding the number of neurons during the design of a deep neural network to maximize performance is not intuitive. Particularly, many application scenarios are real-time and have a strict latency constraint, while conventional neural network optimization methods do not directly change the temporal cost of model inference for latency-critical edge systems. In this work, we propose a latency-oriented neural network learning method to optimize models for high accuracy while fulfilling the latency constraint. For efficiency, we also introduce a universal hardware-customized latency predictor to optimize this procedure to learn a model that satisfies the latency constraint by only a one-shot training process. The experiment results reveal that, compared to state-of-the-art methods, our approach can well-fit the 'hard' latency constraint and achieve high accuracy. Under the same training settings as the original model and satisfying a 34 ms latency constraint on the ImageNet-100 dataset, we reduce GoogLeNet's latency from 40.32 ms to 34 ms with a 0.14% accuracy reduction on the NVIDIA Jetson Nano. When coupled with quantization, our method can be further improved to only 0.04% drop for GoogLeNet. On the NVIDIA Jetson TX2, we compress VGG-19 from 119.98 ms to 34 ms and even improve its accuracy by 0.5%, and we scale GoogLeNet up from 20.27 ms to 34 ms and achieve higher accuracy by 0.78%. We also open source this framework at https://github.com/ntuliuteam/ZeroBN

Figures

Figures reproduced from arXiv: 2607.06922 by Christian Makaya, Di Liu, Hao Kong, Qian Lin, Ravi Subramaniam, Shuo Huai, Weichen Liu.

Figure 1
Figure 1. Figure 1: The workflow of hardware-aware neural network optimization. With [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The relationship between the number of FLOPs and the inference [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: The overview of our model learning framework. The input is a mis [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: The process of our compact learning approach. The input is a redundant model and the output is a compact model that meets the system latency constraint. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: The preprocessing for layer pruning by adding branch layers. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 8
Figure 8. Figure 8: The latency distribution before/after transform. The normal distribu￾tion can improve the accuracy of the latency predictor. the latency of DNN on a certain device. However, the limita￾tions of these methods become obvious throughout the model learning process. Since most devices are black boxes to users, it is difficult to emulate hardware by analyzing its resources and scheduling algorithms. As a result,… view at source ↗
Figure 10
Figure 10. Figure 10: The accuracy w.r.t the variation of k and s values. 3.6. Further discussion As shown in previous sections, our learning is an efficient and effective method to train suitable models for a variety of hardware. Given that BN is included in most DNN models, we believe that our approach has a significant potential for learn￾ing efficient and accurate networks for the emerging edge era. Meanwhile, if a DNN mod… view at source ↗
Figure 10
Figure 10. Figure 10: Due to comparison methods – SFP [6], FPGM [7] and PGMPF [8] train models for 160 epochs on the CIFAR￾10 dataset and 90 epochs on the ImageNet-100 dataset, respec￾4 In this paper, we keep the order of BN layers and convolution layers in the Residual modules [PITH_FULL_IMAGE:figures/full_fig_p011_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Comparison of compact models on CIFAR-10. Baseline means original models without compression. Omit OurLP if no layer pruning. to note that even though PGMPF [8] achieves the highest accu￾racy among all methods with the same training epochs in some cases, its training time is about double that of other methods. 4.3. Accuracy Evaluation We first evaluate our core compact learning in terms of ac￾curacy on th… view at source ↗
Figure 12
Figure 12. Figure 12: Model architecture from layer pruning of VGG-19. [PITH_FULL_IMAGE:figures/full_fig_p013_12.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

50 extracted references · 50 canonical work pages · 10 internal anchors

  1. [1]

    Z. Liu, J. Li, Z. Shen, G. Huang, S. Yan, C. Zhang, Learning efficient convolutional networks through network slimming, in: ICCV , 2017

    work page 2017

  2. [2]

    Pouyanfar, S

    S. Pouyanfar, S. Sadiq, Y . Yan, H. Tian, Y . Tao, M. P. Reyes, M.-L. Shyu, S.-C. Chen, S. S. Iyengar, A survey on deep learning: Algorithms, tech- niques, and applications, ACM Comput. Surv. 51 (5) (2018)

    work page 2018

  3. [3]

    Szegedy, W

    C. Szegedy, W. Liu, Y . Jia, P. Sermanet, S. Reed, D. Anguelov, D. Er- han, V . Vanhoucke, A. Rabinovich, Going deeper with convolutions, in: CVPR, 2015

    work page 2015

  4. [4]

    B. Zoph, Q. V . Le, Neural architecture search with reinforcement learn- ing, arXiv preprint arXiv:1611.01578 (2016)

    work page internal anchor Pith review Pith/arXiv arXiv 2016

  5. [5]

    A. Wan, X. Dai, P. Zhang, Z. He, Y . Tian, S. Xie, B. Wu, M. Yu, T. Xu, K. Chen, et al., Fbnetv2: Differentiable neural architecture search for spatial and channel dimensions, in: Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, 2020, pp. 12965– 12974

    work page 2020

  6. [6]

    Y . He, G. Kang, X. Dong, Y . Fu, Y . Yang, Soft filter pruning for accelerat- ing deep convolutional neural networks, in: IJCAI, 2018, pp. 2234–2240

    work page 2018

  7. [7]

    Y . He, P. Liu, Z. Wang, Z. Hu, Y . Yang, Filter pruning via geometric median for deep convolutional neural networks acceleration, in: CVPR, 2019

    work page 2019

  8. [8]

    L. Cai, Z. An, C. Yang, Y . Yan, Y . Xu, Prior gradient mask guided pruning-aware fine-tuning, in: Proceedings of the AAAI Conference on Artificial Intelligence, V ol. 1, 2022

    work page 2022

  9. [9]

    T. Chen, B. Ji, T. Ding, B. Fang, G. Wang, Z. Zhu, L. Liang, Y . Shi, S. Yi, X. Tu, Only train once: A one-shot neural network training and pruning framework, Advances in Neural Information Processing Systems 34 (2021) 19637–19651

    work page 2021

  10. [10]

    M. Tan, Q. Le, Efficientnet: Rethinking model scaling for convolutional neural networks, in: International Conference on Machine Learning, PMLR, 2019, pp. 6105–6114

    work page 2019

  11. [11]

    H. Kong, D. Liu, X. Luo, W. Liu, R. Subramaniam, Hacscale: Hardware- aware compound scaling for resource-efficient dnns, in: 2022 27th Asia and South Pacific Design Automation Conference (ASP-DAC), IEEE, 2022, pp. 708–713

    work page 2022

  12. [12]

    X. Dai, P. Zhang, B. Wu, H. Yin, F. Sun, Y . Wang, M. Dukhan, Y . Hu, Y . Wu, Y . Jia, P. Vajda, M. Uyttendaele, N. K. Jha, Chamnet: Towards efficient network design through platform-aware model adaptation, in: CVPR, 2019

    work page 2019

  13. [13]

    S. Han, J. Pool, J. Tran, W. Dally, Learning both weights and connections for efficient neural network, in: NIPS, Curran Associates, Inc., 2015, pp. 1135–1143

    work page 2015

  14. [14]

    Z. Liu, M. Sun, T. Zhou, G. Huang, T. Darrell, Rethinking the value of network pruning, in: ICLR, 2019

    work page 2019

  15. [15]

    R. Zhao, Y . Hu, J. Dotzel, C. De Sa, Z. Zhang, Improving neural net- work quantization without retraining using outlier channel splitting, in: International Conference on Machine Learning, 2019

    work page 2019

  16. [16]

    Distilling the Knowledge in a Neural Network

    G. Hinton, O. Vinyals, J. Dean, Distilling the knowledge in a neural net- work, arXiv preprint arXiv:1503.02531 (2015)

    work page internal anchor Pith review Pith/arXiv arXiv 2015

  17. [17]

    Louizos, M

    C. Louizos, M. Welling, D. P. Kingma, Learning sparse neural networks throughl 0 regularization, in: ICLR, 2018

    work page 2018

  18. [18]

    Bjorck, C

    N. Bjorck, C. P. Gomes, B. Selman, K. Q. Weinberger, Understanding batch normalization, in: NIPS, 2018, pp. 7694–7705

    work page 2018

  19. [19]

    J. Choi, P. I.-J. Chuang, Z. Wang, S. Venkataramani, V . Srinivasan, K. Gopalakrishnan, Bridging the accuracy gap for 2-bit quantized neu- ral networks (qnn), arXiv preprint arXiv:1807.06964 (2018)

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  20. [20]

    S. Wu, G. Li, F. Chen, L. Shi, Training and inference with integers in deep neural networks, arXiv preprint arXiv:1802.04680 (2018)

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  21. [21]

    A. Zhou, A. Yao, Y . Guo, L. Xu, Y . Chen, Incremental network quanti- zation: Towards lossless cnns with low-precision weights, arXiv preprint arXiv:1702.03044 (2017)

    work page internal anchor Pith review Pith/arXiv arXiv 2017

  22. [22]

    S. Han, H. Mao, W. J. Dally, Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding, arXiv preprint arXiv:1510.00149 (2015)

    work page internal anchor Pith review Pith/arXiv arXiv 2015

  23. [23]

    W. Sung, S. Shin, K. Hwang, Resiliency of deep neural networks under quantization, arXiv preprint arXiv:1511.06488 (2015)

    work page internal anchor Pith review Pith/arXiv arXiv 2015

  24. [24]

    S. Shin, K. Hwang, W. Sung, Fixed-point performance analysis of recur- rent neural networks, in: 2016 IEEE International Conference on Acous- tics, Speech and Signal Processing (ICASSP), IEEE, 2016, pp. 976–980

    work page 2016

  25. [25]

    Migacz, 8-bit inference with tensorrt, in: GPU technology conference, V ol

    S. Migacz, 8-bit inference with tensorrt, in: GPU technology conference, V ol. 2, 2017, p. 5. 14

    work page 2017

  26. [26]

    Raghu, B

    M. Raghu, B. Poole, J. Kleinberg, S. Ganguli, J. Sohl-Dickstein, On the expressive power of deep neural networks, in: international conference on machine learning, PMLR, 2017, pp. 2847–2854

    work page 2017

  27. [27]

    T.-J. Yang, A. Howard, B. Chen, X. Zhang, A. Go, M. Sandler, V . Sze, H. Adam, Netadapt: Platform-aware neural network adaptation for mo- bile applications, in: ECCV , 2018

    work page 2018

  28. [28]

    B. Wu, X. Dai, P. Zhang, Y . Wang, F. Sun, Y . Wu, Y . Tian, P. Vajda, Y . Jia, K. Keutzer, Fbnet: Hardware-aware efficient convnet design via differentiable neural architecture search, in: CVPR, 2019

    work page 2019

  29. [29]

    Gholami, K

    A. Gholami, K. Kwon, B. Wu, Z. Tai, X. Yue, P. Jin, S. Zhao, K. Keutzer, Squeezenext: Hardware-aware neural network design, in: CVPR Work- shop, 2018

    work page 2018

  30. [30]

    Justus, J

    D. Justus, J. Brennan, S. Bonner, A. S. McGough, Predicting the compu- tational cost of deep learning models, in: 2018 IEEE international confer- ence on big data (Big Data), IEEE, 2018, pp. 3873–3882

    work page 2018

  31. [31]

    L. L. Zhang, S. Han, J. Wei, N. Zheng, T. Cao, Y . Yang, Y . Liu, nn-meter: towards accurate latency prediction of deep-learning model inference on diverse edge devices, in: Proceedings of the 19th Annual International Conference on Mobile Systems, Applications, and Services, 2021, pp. 81–93

    work page 2021

  32. [32]

    S. Huai, L. Zhang, D. Liu, W. Liu, R. Subramaniam, Zerobn: Learn- ing compact neural networks for latency-critical edge systems, in: 2021 58th ACM/IEEE Design Automation Conference (DAC), IEEE, 2021, pp. 151–156

    work page 2021

  33. [33]

    H. Li, A. Kadav, I. Durdanovic, H. Samet, H. P. Graf, Pruning filters for efficient convnets, arXiv preprint arXiv:1608.08710 (2016)

    work page internal anchor Pith review Pith/arXiv arXiv 2016

  34. [34]

    Pruning Convolutional Neural Networks for Resource Efficient Inference

    P. Molchanov, S. Tyree, T. Karras, T. Aila, J. Kautz, Pruning convo- lutional neural networks for resource efficient inference, arXiv preprint arXiv:1611.06440 (2016)

    work page internal anchor Pith review Pith/arXiv arXiv 2016

  35. [35]

    Goh, Why momentum really works

    G. Goh, Why momentum really works. distill (2017)

    work page 2017

  36. [36]

    Dudziak, T

    L. Dudziak, T. Chau, M. Abdelfattah, R. Lee, H. Kim, N. Lane, Brp- nas: Prediction-based nas using gcns, Advances in Neural Information Processing Systems 33 (2020) 10480–10490

    work page 2020

  37. [37]

    Hecht-Nielsen, Theory of the backpropagation neural network, in: Neural networks for perception, Elsevier, 1992, pp

    R. Hecht-Nielsen, Theory of the backpropagation neural network, in: Neural networks for perception, Elsevier, 1992, pp. 65–93

    work page 1992

  38. [38]

    De Villiers, E

    J. De Villiers, E. Barnard, Backpropagation neural nets with one and two hidden layers, IEEE transactions on neural networks 4 (1) (1993) 136– 141

    work page 1993

  39. [39]

    Lee, C.-Y

    E. Lee, C.-Y . Lee, Neuralscale: Efficient scaling of neurons for resource- constrained deep neural networks, in: Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition, 2020, pp. 1478– 1487

    work page 2020

  40. [40]

    Krizhevsky, Learning multiple layers of features from tiny images (2009)

    A. Krizhevsky, Learning multiple layers of features from tiny images (2009)

    work page 2009

  41. [41]

    Very Deep Convolutional Networks for Large-Scale Image Recognition

    K. Simonyan, A. Zisserman, Very deep convolutional networks for large- scale image recognition (2015). arXiv:1409.1556

    work page internal anchor Pith review Pith/arXiv arXiv 2015

  42. [42]

    Russakovsky, J

    O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, A. C. Berg, L. Fei-Fei, ImageNet Large Scale Visual Recognition Challenge, IJCV 115 (3) (2015) 211–252

    work page 2015

  43. [43]

    Bianco, R

    S. Bianco, R. Cadene, L. Celona, P. Napoletano, Benchmark analysis of representative deep neural network architectures, IEEE access 6 (2018) 64270–64277

    work page 2018

  44. [44]

    Abadi, et al., TensorFlow: Large-scale machine learning on heteroge- neous systems, software available from tensorflow.org (2015)

    M. Abadi, et al., TensorFlow: Large-scale machine learning on heteroge- neous systems, software available from tensorflow.org (2015)

    work page 2015

  45. [45]

    Z. Cao, M. Long, J. Wang, P. S. Yu, Hashnet: Deep learning to hash by continuation, in: ICCV , 2017

    work page 2017

  46. [46]

    Cakir, K

    F. Cakir, K. He, S. Sclaroff, Hashing with binary matrix pursuit, in: ECCV , 2018

    work page 2018

  47. [47]

    K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recog- nition, in: CVPR, 2016

    work page 2016

  48. [48]

    Huang, Z

    G. Huang, Z. Liu, L. van der Maaten, K. Q. Weinberger, Densely con- nected convolutional networks, in: CVPR, 2017

    work page 2017

  49. [49]

    Paszke, S

    A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, S. Chintala, Pytorch: An imperative style, high- performance deep learning library, in: NIPS, 2019, pp. 8024–8035

    work page 2019

  50. [50]

    H. Mao, et al., Exploring the granularity of sparsity in convolutional neu- ral networks, in: Proceedings of the IEEE Conference on Computer Vi- sion and Pattern Recognition Workshops, 2017, pp. 13–20. 15

    work page 2017