Pith. sign in

REVIEW 5 major objections 5 minor 58 references

Lightweight Design and Optimization methods for DCNNs: Progress and Futures

T0 review · 5 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper presents a taxonomy of lightweight deep convolutional networks: design efficiency into the architecture or compress an existing model, with trade-offs for hardware deployment.

desk verdict A readable survey whose reference-to-concept mapping is systematically unreliable; the taxonomy is fine but the pointer errors undercut its purpose. read the letter →

arxiv 2412.16886 v1 pith:OM2AP7LQ submitted 2024-12-22 cs.CV

Hanhua Long , Wenbin Bi , Jian Sun This is my paper · ORCID
classification cs.CV
keywords lightweightneuralnetworksdeepconvolutionalarchitecturesearchmodelcompressionpruningquantizationknowledgedistillationlow-rankdecomposition
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Lightweight deep convolutional neural networks are a response to a practical bottleneck: high-accuracy vision models demand storage and computation that mobile and embedded hardware cannot afford. This review organizes the field into two complementary strategies. The first builds efficiency into the architecture itself, through small or factored convolutions, cheap operations, structural reparameterization, and automated neural architecture search. The second compresses an already-trained model through pruning, quantization, low-rank decomposition, and knowledge distillation. The paper's aim is to give practitioners a structured map of these methods, their trade-offs, and the open problems that remain.

What carries the argument

The organizing device is a taxonomy of lightweighting: manual architectural design, AutoML-driven neural architecture search, and model compression. Within that taxonomy, the specific mechanisms carrying the argument are the computational shortcuts — depthwise separable convolution (one filter per input channel followed by a $1\times1$ channel mixer), group convolution (channels split into $g$ groups, cutting parameters to $1/g$), $1\times1$ convolution as a channel mixer, structural reparameterization (training a multi-branch net that algebraically collapses into a single-branch inference net), and the four compression operations (pruning, quantization, low-rank decomposition, knowledge distillation). The taxonomy does the work of placing every reviewed method on a spectrum from changing how the network computes to changing what the network stores.

What would settle it

If a mainstream lightweighting method can be identified that is neither a change to the network architecture nor a compression of an existing model, the review's central dichotomy fails; a literature sweep of recent efficient-CNN papers would settle this by checking whether any method reduces inference cost purely through compiler-level scheduling with no change to weights or topology.

Watch

Extended reading notes

Core claim

The review's central claim is that lightweighting DCNNs is not one technique but a design space with two poles: architectural design and model compression. On the design side, it describes how depthwise separable convolutions cut computation to $1/k^2$ of a standard $k\times k$ convolution, how group convolutions divide parameters by the number of groups, how $1\times1$ convolutions shrink channels, how cheaper operations such as AdderNet replace multiplications with additions, and how structural reparameterization (RepVGG) lets a multi-branch training network collapse into a single-branch inference network. On the compression side, it covers pruning, weight quantization, low-rank approximation, and knowledge distillation, and notes the typical costs of each, such as accuracy loss at extreme quantization. It also maps neural architecture search into search space, search strategy, and performance evaluation, and cites system-algorithm co-design (MCUNet) as the route that pushes lightweight models onto microcontrollers.

Load-bearing premise

The whole review's usefulness rests on the assumption that each cited method is correctly attributed and faithfully described, so that the map readers take away matches the actual landscape of the field.

Editorial extensions

If this is right

  • A practitioner can choose between building efficiency in from the start and squeezing an existing model afterward; the right choice depends on whether retraining is affordable and whether the target hardware rewards sparsity or structured operations.
  • Structured pruning and reparameterized single-branch inference are hardware-friendly because they keep the final network dense and regular, while unstructured pruning and extreme low-bit quantization may win on paper but lose on real edge hardware.
  • The three-way decomposition of NAS into search space, search strategy, and performance evaluation implies that progress in any one component, such as cheaper evaluation, should accelerate the whole pipeline.
  • System-algorithm co-design, as illustrated by MCUNet's TinyNAS and TinyEngine, extends lightweight design from network structure to memory scheduling, which is what matters when memory is two to three orders of magnitude smaller than a smartphone's.
  • Combining compression techniques (prune, then quantize, then distill) is presented as the natural next step, since each method attacks a different source of redundancy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension of this taxonomy is a benchmark protocol that measures accuracy under a joint memory-latency-energy budget on several hardware targets; the review's own caveats imply that FLOP counts alone would misrank methods.
  • The boundary between design and compression is already dissolving: structural reparameterization is a design-time trick that behaves like compression, and NAS can search over compressed cells, so future surveys may treat the distinction as organizational rather than fundamental.
  • An implication the authors leave implicit is that knowledge distillation and quantization are natural partners for neural architecture search: the searched student can be trained against a large teacher and then quantized, compounding the gains.
  • The review's outlook implies that lightweight design will become increasingly hardware-specific, so a method that wins on one chip may lose on another; this could be tested by evaluating the same compressed model across CPU, GPU, and microcontroller targets.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. This paper is a survey of lightweight DCNN design and model compression. It organizes the field into manual architectural design (kernel-size choices, depthwise/group convolutions, cheap operations such as AdderNet and GhostNet, and structural reparameterization), AutoML-driven neural architecture search (search spaces, search strategies, and evaluation strategies), and four compression families (pruning, quantization, low-rank approximation, and knowledge distillation). The paper closes with general observations about limitations and future research directions. The stated goal is to provide practitioners with a map of the main approaches and their trade-offs.

Significance. If its citations and quantitative statements were accurate, this survey would fill a useful niche: it covers manual design, NAS, and compression in one place and provides comparative tables of model sizes and compression-method trade-offs. The breadth is good, and the figures are generally clear. However, the paper's core value as a reliable map from named techniques to primary sources is currently undermined by a dense pattern of citation mismatches and several numeric errors; a reader who follows the references to locate original methods will frequently be misrouted. These issues are fixable, but they are central to the genre and must be corrected before the survey can be accepted.

major comments (5)
  1. [Section 2.1] The MobileNet sentence in this section states that depthwise separable convolutions reduce computation by 'approximately 12% compared to standard 3×3 convolutions.' As worded, this says the reduction is 12%, leaving 88% of the cost. The correct statement is that the depthwise-separable cost is about 1/K^2 + 1/C_out of a standard convolution, which for typical channel counts is roughly 11–12% of the original cost (i.e., a reduction of about 88–89%). In addition, Section 2.2 later says the cost is 'only 1/k^2' of a regular convolution, which omits the pointwise convolution term and is inconsistent with the Section 2.1 wording. Please correct the direction and magnitude and reconcile the two statements.
  2. [Section 3.3, Table 2] The mapping between the four named evaluation strategies and the cited references in Table 2 is incorrect throughout. References [44-46] are object detection, Inception-v3, and EfficientNet papers; references [47-49] are a content-based image classification paper, a CNN acceleration paper, and a structured-sparsity paper; references [50-52] are FBNet, kernel-based progressive distillation for AdderNet, and energy-aware pruning; and references [53-55] are a knowledge-distillation paper and two CNN acceleration papers. None of these groups support the labels 'low fidelity,' 'surrogate model,' 'network morphism,' or 'one-time search.' Since Table 2 is the only detailed presentation of performance-evaluation strategies in this section, the survey does not currently provide the promised map of NAS evaluation methods; the table and surrounding text need to be rebuilt with correct primary references.
  3. [Introduction and Section 3.1.2] The MnasNet work is misattributed in two places. The Introduction credits 'Mobile Neural Architecture Search (MNAS)' to Tan et al. [46], but reference [46] is the EfficientNet paper. Section 3.1.2 says that Zoph et al. [59] 'proposed the search space for MNASNet,' but reference [59] is the NASNet paper. MnasNet is the platform-aware search of Tan et al. (2018), and its search space is not Zoph et al.'s. These misattributions misroute readers on a central result in lightweight NAS, and the text around Figure 6 must be corrected.
  4. [Section 3.2.1] This paragraph describes Bello et al. [2] as generating neural network architectures with a recurrent controller, but reference [2] is the paper 'Neural Optimizer Search with Reinforcement Learning,' which searches for optimizers, not architectures. The description does not match the cited paper. The authors should either replace this passage with the correct reference for the described architecture-search method (e.g., Zoph and Le, 2017) or rewrite the paragraph to describe optimizer search.
  5. [Section 2.3] The method of Xu et al. [51] is renamed 'Progressive Kernel Knowledge Distillation (PKKD),' but the reference's own title in the bibliography is 'Kernel Based Progressive Distillation for Adder Neural Networks.' The text should use the name from the primary source, and the description of the mechanism should be checked against the original paper to ensure it does not invent extra components such as projecting weights in a way not described there.
minor comments (5)
  1. [Section 2.1] The sentence 'Szegedy et al. [43] introduced 1×1 kernels in Inception V3' is incorrect: reference [43] is the GoogLeNet paper ('Going Deeper with Convolutions'), where 1×1 kernels were introduced in the Inception family, while Inception V3 is reference [45]. Please correct the citation and model name.
  2. [Section 4.4 and Figure 9] The text says the student model's softmax output is used as the 'hard target.' The hard target should be the ground-truth label; the student's softmax output is the prediction being trained. Please fix this terminology.
  3. [Section 2.2] The statement that the computational cost of depthwise separable convolutions is 'only 1/k^2' of a regular convolution neglects the pointwise 1×1 convolution, so it is at best a limiting-case statement. Add a qualifier such as 'for the depthwise part' or 'approximately.'
  4. [Section 3.3] The text uses 'one-shot search' while Table 2 uses 'One-time Search'; please standardize the terminology and verify that the intended concept is clearly defined.
  5. [General] The manuscript needs careful copyediting for grammar and article usage; several sentences, such as 'This approach implies a correspondence between the training structures...' in Section 2.4, are difficult to parse. A professional language edit would improve readability.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper is a literature survey that derives nothing from its own claims or fitted parameters.

full rationale

This manuscript is a review of lightweight DCNN design and compression methods. It contains no derivation chain, predictive model, fitted parameter, or formal theorem that could reduce to its own inputs. The paper's content consists of descriptions and classifications of external published methods, and the central claim is only that the paper reviews the field. That claim is definitionally satisfied by the text itself rather than being a derived result. The identified citation-to-concept misattributions, such as crediting the MNASNet search space to Zoph et al. [59] and renaming Xu et al.'s method as PKKD, are accuracy and provenance defects, not circular reasoning: they do not make any conclusion equivalent to an assumption of the paper. There are no self-citations that carry a load-bearing argument, and no result is imported from the authors' own prior work to forbid alternatives. Accordingly, the circularity score is 0.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The review introduces no new parameters or entities. Its output depends on the accuracy and representativeness of the prior work it summarizes, which is only partially assured given the citation and numerical errors noted.

assumptions (3)
  • domain assumption The descriptions of the surveyed prior works are faithful to the original publications.
    The review's central service is accurate summarization; Sections 2.3 and 3.1.2 misdescribe two cited methods, so this assumption is partially violated.
  • standard math Standard cost formulas for convolutional operations, such as the claim that depthwise separable convolutions cost about 1/k^2 of regular convolutions, are correct.
    Used in Section 2.2; the 1/k^2 formula is an approximation that ignores the pointwise convolution's cost, and the related '12%' statement in Section 2.1 appears incorrect.
  • domain assumption The cited references provide a representative sample of the field of lightweight DCNNs.
    The review claims to examine recent research progress, but no systematic search or inclusion criteria are given, so representativeness is assumed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Lightweight Design and Optimization methods for DCNNs: Progress and Futures." pith.science (2026). https://pith.science/paper/OM2AP7LQ

@misc{pith2026241216886,
  author       = {Pith},
  title        = {Pith review of: Lightweight Design and Optimization methods for DCNNs: Progress and Futures},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OM2AP7LQ}},
  note         = {Machine review of arXiv:2412.16886}
}
read the original abstract

Lightweight design, as a key approach to mitigate disparity between computational requirements of deep learning models and hardware performance, plays a pivotal role in advancing application of deep learning technologies on mobile and embedded devices, alongside rapid development of smart home, telemedicine, and autonomous driving. With its outstanding feature extracting capabilities, Deep Convolutional Neural Networks (DCNNs) have demonstrated superior performance in computer vision tasks. However, high computational costs and large network architectures severely limit the widespread application of DCNNs on resource-constrained hardware platforms such as smartphones, robots, and IoT devices. This paper reviews lightweight design strategies for DCNNs and examines recent research progress in both lightweight architectural design and model compression. Additionally, this paper discusses current limitations in this field of research and propose prospects for future directions, aiming to provide valuable guidance and reflection for lightweight design philosophy on deep neural networks in the field of computer vision.

Figures

Figures reproduced from arXiv: 2412.16886 by the authors.

Figure 1
Figure 1. Depthwise Convolution serves on spatial feature extractor, operating on each channel independently. Meanwhile, Pointwise Convolution acts as a channel mixer, integrating information across various channels. applications. Peng et al. [35], on the other hand, suggested using two layers of 1×k and k×1 convolutions in place of k×k convolutions, significantly lowering computational requirements while maintaining a large … view at source ↗
Figure 2
Figure 2. Group Convolution employs a set of kernels, each operating on its respective segment of the input. feature extraction phase. Consequently, researchers actively explore innovations in convolutional architectures aimed at reducing parameters meanwhile maintain accuracy. Depthwise separable convolutions and group convolutions, as two pivotal strategies, have significantly influenced and been widely implemented in this … view at source ↗
Figure 3
Figure 3. In RepVGG, a multi-branch structure with residual connection is employed during the training phase, whereas a chain structure equipped with 3×3 kernel is utilized for inference. Structural reparameterization can not only accelerate inference but also enable feature reuse without concatenation operations. Chen et al. [5] proposed a hardware-efficient RepGhost module. They noted that although traditional concatenation… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: General Process of Neural Architecture Search. 1. Search space is the set of candidate structures available for selection in optimization problem of neural network architecture, that is, the domain of solutions. It encompasses all possible network architectures, from s…
Figure 5
Figure 5. Figure 5: Brief showcase of chain structure and multi-branch structure. residual connections to the chain structure, increasing the flexibility of the search space and expanding the size of the chain-structured search space. The simplicity of chain structures often precludes the…
Figure 6
Figure 6. Figure 6: In MNASNet, different block may have diverse internal structures, resulting in a local-based search space. Zoph et al. [59] were the first to explore block-based search spaces and proposed the search space for MNASNet. As shown in [PITH_FULL_IMAGE:figures/full_fig_p00…
Figure 7
Figure 7. Figure 7: General process of Reinforcement Learning. architectures. This section introduces two primary search strategies: reinforcement learning-based and gradient-based methods. 3.2.1. Search Strategy Based on Reinforcement Learning Reinforcement learning is a branch of machin…
Figure 8
Figure 8. Figure 8: Brief showcase of Model Pruning. Model pruning primarily divides into structured pruning and unstructured pruning. Unstructured pruning removes insignificant neurons, and correspondingly, the connections between pruned neurons and other neurons are ignored during compu…
Figure 9
Figure 9. Figure 9: General process of Knowledge Distillation. cognition, proving that it is still possible to effectively alleviate the computational burden of the model while maintaining high-performance standards. 4.4. Knowledge Distillation Knowledge distillation was initially propose…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

58 extracted references · 28 canonical work pages

  1. [46]

    Tan,M.,Le,Q.,2019. EfficientNet:Rethinkingmodelscalingforconvolutionalneuralnetworks,in:Chaudhuri,K.,Salakhutdinov,R.(Eds.), Proceedingsofthe36thInternationalConferenceonMachineLearning,PMLR.pp.6105–6114.URL: https://proceedings.mlr.press/ v97/tan19a.html

    work page 2019

  2. [59]

    Learning transferable architectures for scalable image recognition, in: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    Zoph, B., Vasudevan, V., Shlens, J., Le, Q.V., 2018. Learning transferable architectures for scalable image recognition, in: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8697–8710. doi:10.1109/CVPR.2018.00907. Hanhua Long et al.:Preprint submitted to Elsevier Page 18 of 18

    arXiv 2018

  3. [2]

    Neural optimizer search with reinforcement learning, in: Precup, D., Teh, Y.W

    Bello, I., Zoph, B., Vasudevan, V., Le, Q.V., 2017. Neural optimizer search with reinforcement learning, in: Precup, D., Teh, Y.W. (Eds.), Proceedings of the 34th International Conference on Machine Learning, PMLR. pp. 459–468. URL:https://proceedings.mlr.press/ v70/bello17a.html

    work page 2017

  4. [51]

    Xu,Y.,Xu,C.,Chen,X.,Zhang,W.,Xu,C.,Wang,Y.,2020. Kernelbasedprogressivedistillationforadderneuralnetworks,in:Proceedings of the 34th International Conference on Neural Information Processing Systems, Curran Associates Inc., Red Hook, NY, USA

    work page 2020

  5. [1]

    Designingneuralnetworkarchitecturesusingreinforcementlearning

    Baker,B.,Gupta,O.,Naik,N.,Raskar,R.,2016. Designingneuralnetworkarchitecturesusingreinforcementlearning. CoRRabs/1611.02167. URL: http://arxiv.org/abs/1611.02167, arXiv:1611.02167

    arXiv 2016

  6. [3]

    Modelcompression,in:Proceedingsofthe12thACMSIGKDDInternationalConference on Knowledge Discovery and Data Mining, Association for Computing Machinery, New York, NY, USA

    Bucilˇa,C.,Caruana,R.,Niculescu-Mizil,A.,2006. Modelcompression,in:Proceedingsofthe12thACMSIGKDDInternationalConference on Knowledge Discovery and Data Mining, Association for Computing Machinery, New York, NY, USA. p. 535–541. URL:https: //doi.org/10.1145/1150402.1150464, doi:10.1145/1150402.1150464

    arXiv 2006

  7. [4]

    Proxylessnas: Direct neural architecture search on target task and hardware

    Cai, H., Zhu, L., Han, S., 2018. Proxylessnas: Direct neural architecture search on target task and hardware. CoRR abs/1812.00332. URL: http://arxiv.org/abs/1812.00332, arXiv:1812.00332

    arXiv 2018

  8. [5]

    Repghost: A hardware-efficient ghost module via re-parameterization

    Chen, C., Guo, Z., Zeng, H., Xiong, P., Dong, J., 2022. Repghost: A hardware-efficient ghost module via re-parameterization. ArXiv abs/2211.06088

    arXiv 2022

Show all 58 references
  1. [7]

    Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs

    Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L., 2018a. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Transactions on Pattern Analysis and Machine Intelligence 40, 834–848. doi:10. 1109/...

  2. [8]

    Darkrank: Accelerating deep metric learning via cross sample similarities transfer

    Chen, Y., Wang, N., Zhang, Z., 2018b. Darkrank: Accelerating deep metric learning via cross sample similarities transfer. Proceedings of the AAAI Conference on Artificial Intelligence 32. URL:https://ojs.aaai.org/index.php/AAAI/article/view/11783, doi:10. 1609/aaai.v32i1.11783

  3. [9]

    Sobolev training for neural networks, in: Proceedings of the 31st International Conference on Neural Information Processing Systems, Curran Associates Inc., Red Hook, NY, USA

    Czarnecki, W.M., Osindero, S., Jaderberg, M., Swirszcz, G., Pascanu, R., 2017. Sobolev training for neural networks, in: Proceedings of the 31st International Conference on Neural Information Processing Systems, Curran Associates Inc., Red Hook, NY, USA. p. 4281–4290. Hanhua L...

    2017

  4. [10]

    Denton, E., Zaremba, W., Bruna, J., LeCun, Y., Fergus, R., 2014. Exploiting linear structure within convolutional networks for efficient evaluation,in:Proceedingsofthe27thInternationalConferenceonNeuralInformationProcessingSystems-Volume1,MITPress,Cambridge, MA, USA. p. 1269–1277

    2014

  5. [11]

    Repvgg:Makingvgg-styleconvnetsgreatagain,in:2021IEEE/CVFConference on Computer Vision and Pattern Recognition (CVPR), pp

    Ding,X.,Zhang,X.,Ma,N.,Han,J.,Ding,G.,Sun,J.,2021. Repvgg:Makingvgg-styleconvnetsgreatagain,in:2021IEEE/CVFConference on Computer Vision and Pattern Recognition (CVPR), pp. 13728–13737. doi:10.1109/CVPR46437.2021.01352

    2021

  6. [12]

    Ghostnet: More features from cheap operations, in: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp

    Han, K., Wang, Y., Tian, Q., Guo, J., Xu, C., Xu, C., 2020. Ghostnet: More features from cheap operations, in: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1577–1586. doi:10.1109/CVPR42600.2020.00165

    2020

  7. [13]

    Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding, in: International Conference on Learning Representations

    Han, S., Mao, H., Dally, W.J., 2016. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding, in: International Conference on Learning Representations

    2016

  8. [14]

    Deep residual learning for image recognition, in: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp

    He, K., Zhang, X., Ren, S., Sun, J., 2016. Deep residual learning for image recognition, in: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778. doi:10.1109/CVPR.2016.90

    2016 doi

  9. [15]

    He,Y.,Lin,J.,Liu,Z.,Wang,H.,Li,L.J.,Han,S.,2018.Amc:Automlformodelcompressionandaccelerationonmobiledevices,in:Computer Vision – ECCV 2018: 15th European Conference, Munich, Germany, September 8–14, 2018, Proceedings, Part VII, Springer-Verlag, Berlin, Heidelberg. p. 815–832. ...

    2018 doi

  10. [16]

    Distilling the knowledge in a neural network

    Hinton, G.E., Vinyals, O., Dean, J., 2015. Distilling the knowledge in a neural network. ArXiv abs/1503.02531

    2015 arXiv

  11. [17]

    Searchingformobilenetv3,in:2019IEEE/CVFInternationalConferenceonComputerVision(ICCV),pp.1314–1324

    Howard, A., Sandler, M., Chen, B., Wang, W., Chen, L.C., Tan, M., Chu, G., Vasudevan, V., Zhu, Y., Pang, R., Adam, H., Le, Q., 2019. Searchingformobilenetv3,in:2019IEEE/CVFInternationalConferenceonComputerVision(ICCV),pp.1314–1324. doi: 10.1109/ICCV. 2019.00140

    2019

  12. [18]

    Mobilenets: Efficient convolutional neural networks for mobile vision applications

    Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., Adam, H., 2017. Mobilenets: Efficient convolutional neural networks for mobile vision applications. ArXiv abs/1704.04861

    2017 arXiv

  13. [19]

    Anovelchannelpruningmethodfordeepneuralnetworkcompression

    Hu,Y.,Sun,S.,Li,J.,Wang,X.,Gu,Q.,2018. Anovelchannelpruningmethodfordeepneuralnetworkcompression. CoRRabs/1805.11394. URL: http://arxiv.org/abs/1805.11394, arXiv:1805.11394

    2018 arXiv

  14. [20]

    Densely connected convolutional networks, in: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp

    Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q., 2017. Densely connected convolutional networks, in: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2261–2269. doi:10.1109/CVPR.2017.243

    2017 doi

  15. [21]

    Binarized neural networks, in: Proceedings of the 30th International Conference on Neural Information Processing Systems, Curran Associates Inc., Red Hook, NY, USA

    Hubara, I., Courbariaux, M., Soudry, D., El-Yaniv, R., Bengio, Y., 2016. Binarized neural networks, in: Proceedings of the 30th International Conference on Neural Information Processing Systems, Curran Associates Inc., Red Hook, NY, USA. p. 4114–4122

    2016

  16. [22]

    Squeezenet: Alexnet-level accuracy with 50x fewer parameters and <1mb model size

    Iandola, F.N., Moskewicz, M.W., Ashraf, K., Han, S., Dally, W.J., Keutzer, K., 2016. Squeezenet: Alexnet-level accuracy with 50x fewer parameters and <1mb model size. CoRR abs/1602.07360. URL:http://arxiv.org/abs/1602.07360, arXiv:1602.07360

    2016 arXiv

  17. [23]

    Compression of deep convolutional neural networks for fast and low power mobile applications, in: International Conference on Learning Representations

    Kim, Y.D., Park, E., Yoo, S., Choi, T., Yang, L., Shin, D., 2016. Compression of deep convolutional neural networks for fast and low power mobile applications, in: International Conference on Learning Representations

    2016

  18. [24]

    Krizhevsky,A.,Sutskever,I.,Hinton,G.E.,2012. Imagenetclassificationwithdeepconvolutionalneuralnetworks,in:Proceedingsofthe25th InternationalConferenceonNeuralInformationProcessingSystems-Volume1,CurranAssociatesInc.,RedHook,NY,USA.p.1097–1105

    2012

  19. [25]

    Speeding-up convolutional neural networks using fine-tuned cp- decomposition, in: International Conference on Learning Representations

    Lebedev, V., Ganin, Y., Rakhuba, M., Oseledets, I., Lempitsky, V., 2015. Speeding-up convolutional neural networks using fine-tuned cp- decomposition, in: International Conference on Learning Representations

    2015

  20. [26]

    Gradient-based learning applied to document recognition

    Lecun, Y., Bottou, L., Bengio, Y., Haffner, P., 1998. Gradient-based learning applied to document recognition. Proceedings of the IEEE 86, 2278–2324. doi:10.1109/5.726791

    1998 doi

  21. [27]

    Optimal brain damage, in: Touretzky, D

    LeCun, Y., Denker, J., Solla, S., 1989. Optimal brain damage, in: Touretzky, D. (Ed.), Advances in Neural Informa- tion Processing Systems, Morgan-Kaufmann. URL: https://proceedings.neurips.cc/paper_files/paper/1989/file/ 6c9882bbac1c7093bd25041881277658-Paper.pdf

    1989

  22. [28]

    Memory-efficient patch-based inference for tiny deep learning, in: Ranzato, M., Beygelzimer, A., Dauphin, Y., Liang, P., Vaughan, J.W

    Lin, J., Chen, W.M., Cai, H., Gan, C., Han, S., 2021. Memory-efficient patch-based inference for tiny deep learning, in: Ranzato, M., Beygelzimer, A., Dauphin, Y., Liang, P., Vaughan, J.W. (Eds.), Advances in Neural Information Processing Sys- tems, Curran Associates, Inc.. pp...

    2021

  23. [29]

    Lin, J., Chen, W.M., Lin, Y., cohn, j., Gan, C., Han, S., 2020. Mcunet: Tiny deep learning on iot devices, in: Larochelle, H., Ranzato, M., Hadsell,R.,Balcan,M.,Lin,H.(Eds.),AdvancesinNeuralInformationProcessingSystems,CurranAssociates,Inc..pp.11711–11722. URL: https://proceed...

    2020

  24. [30]

    Ternary weight networks, in: ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp

    Liu, B., Li, F., Wang, X., Zhang, B., Yan, J., 2023. Ternary weight networks, in: ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1–5. doi:10.1109/ICASSP49357.2023.10094626

    2023

  25. [31]

    Hierarchicalrepresentationsforefficientarchitecturesearch

    Liu,H.,Simonyan,K.,Vinyals,O.,Fernando,C.,Kavukcuoglu,K.,2017. Hierarchicalrepresentationsforefficientarchitecturesearch. CoRR abs/1711.00436. URL: http://arxiv.org/abs/1711.00436, arXiv:1711.00436

    2017 arXiv

  26. [32]

    DARTS:differentiablearchitecturesearch

    Liu,H.,Simonyan,K.,Yang,Y.,2018. DARTS:differentiablearchitecturesearch. CoRRabs/1806.09055. URL: http://arxiv.org/abs/ 1806.09055, arXiv:1806.09055

    2018 arXiv

  27. [33]

    An entropy-based pruning method for cnn compression

    Luo, J., Wu, J., 2017. An entropy-based pruning method for cnn compression. CoRR abs/1706.05791. URL:http://arxiv.org/abs/ 1706.05791, arXiv:1706.05791

    2017 arXiv

  28. [34]

    Extremenetworkcompressionviafiltergroupapproximation,in:Ferrari,V.,Hebert, M., Sminchisescu, C., Weiss, Y

    Peng,B.,Tan,W.,Li,Z.,Zhang,S.,Xie,D.,Pu,S.,2018. Extremenetworkcompressionviafiltergroupapproximation,in:Ferrari,V.,Hebert, M., Sminchisescu, C., Weiss, Y. (Eds.), Computer Vision – ECCV 2018, Springer International Publishing, Cham. pp. 307–323

    2018

  29. [35]

    Largekernelmatters—improvesemanticsegmentationbyglobalconvolutionalnetwork, in: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp

    Peng,C.,Zhang,X.,Yu,G.,Luo,G.,Sun,J.,2017. Largekernelmatters—improvesemanticsegmentationbyglobalconvolutionalnetwork, in: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1743–1751. doi:10.1109/CVPR.2017.189

    2017 doi

  30. [36]

    Xnor-net: Imagenet classification using binary convolutional neural networks, in: Leibe, B., Matas, J., Sebe, N., Welling, M

    Rastegari, M., Ordonez, V., Redmon, J., Farhadi, A., 2016. Xnor-net: Imagenet classification using binary convolutional neural networks, in: Leibe, B., Matas, J., Sebe, N., Welling, M. (Eds.), Computer Vision – ECCV 2016, Springer International Publishing, Cham. pp. 525–542

    2016

  31. [37]

    AutoML-zero: Evolving machine learning algorithms from scratch, in: III, H.D., Singh, A

    Real, E., Liang, C., So, D., Le, Q., 2020. AutoML-zero: Evolving machine learning algorithms from scratch, in: III, H.D., Singh, A. (Eds.), Proceedingsofthe37thInternationalConferenceonMachineLearning,PMLR.pp.8007–8019.URL: https://proceedings.mlr.press/ v119/real20a.html. Han...

    2020

  32. [38]

    Mobilenetv2: Inverted residuals and linear bottlenecks, in: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C., 2018. Mobilenetv2: Inverted residuals and linear bottlenecks, in: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4510–4520. doi:10.1109/CVPR.2018.00474

    2018

  33. [39]

    Efficient acceleration of deep learning inference on resource-constrained edge devices: A review

    Shuvo, M.M.H., Islam, S.K., Cheng, J., Morshed, B.I., 2023. Efficient acceleration of deep learning inference on resource-constrained edge devices: A review. Proceedings of the IEEE 111, 42–91. doi:10.1109/JPROC.2022.3226481

    2023

  34. [40]

    Very deep convolutional networks for large-scale image recognition, in: International Conference on Learning Representations, p

    Simonyan, K., Zisserman, A., 2015. Very deep convolutional networks for large-scale image recognition, in: International Conference on Learning Representations, p. 1–14

    2015

  35. [41]

    Addersr: Towards energy efficient image super-resolution, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp

    Song, D., Wang, Y., Chen, H., Xu, C., Xu, C., Tao, D., 2021. Addersr: Towards energy efficient image super-resolution, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 15648–15657

    2021

  36. [42]

    Inception-v4, inception-resnet and the impact of residual connections on learning, in: Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, AAAI Press

    Szegedy, C., Ioffe, S., Vanhoucke, V., Alemi, A.A., 2017. Inception-v4, inception-resnet and the impact of residual connections on learning, in: Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, AAAI Press. p. 4278–4284

    2017

  37. [43]

    Going deeper with convolutions,in:2015IEEEConferenceonComputerVisionandPatternRecognition(CVPR),pp.1–9

    Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A., 2015. Going deeper with convolutions,in:2015IEEEConferenceonComputerVisionandPatternRecognition(CVPR),pp.1–9. doi: 10.1109/CVPR.2015.7298594

    2015

  38. [44]

    Szegedy, C., Toshev, A., Erhan, D., 2013. Deep neural networks for object detection, in: Proceedings of the 26th International Conference on Neural Information Processing Systems - Volume 2, Curran Associates Inc., Red Hook, NY, USA. p. 2553–2561

    2013

  39. [45]

    Rethinking the inception architecture for computer vision, in: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp

    Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z., 2016. Rethinking the inception architecture for computer vision, in: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2818–2826. doi:10.1109/CVPR.2016.308

    2016 doi

  40. [47]

    Image classification for content-based indexing

    Vailaya, A., Figueiredo, M., Jain, A., Zhang, H.J., 2001. Image classification for content-based indexing. IEEE Transactions on Image Processing 10, 117–130. doi:10.1109/83.892448

    2001 doi

  41. [48]

    Acceleratingconvolutionalneuralnetworksformobileapplications,in:Proceedingsofthe24thACMInternational Conferenceon Multimedia,AssociationforComputingMachinery, NewYork,NY,USA

    Wang,P.,Cheng,J.,2016. Acceleratingconvolutionalneuralnetworksformobileapplications,in:Proceedingsofthe24thACMInternational Conferenceon Multimedia,AssociationforComputingMachinery, NewYork,NY,USA. p.541–545. URL: https://doi.org/10.1145/ 2964284.2967280, doi:10.1145/2964284.2967280

    2016

  42. [49]

    Wen, W., Wu, C., Wang, Y., Chen, Y., Li, H., 2016. Learning structured sparsity in deep neural networks, in: Proceedings of the 30th International Conference on Neural Information Processing Systems, Curran Associates Inc., Red Hook, NY, USA. p. 2082–2090

    2016

  43. [50]

    Fbnet: Hardware-aware efficient convnet design via differentiable neural architecture search, in: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp

    Wu, B., Dai, X., Zhang, P., Wang, Y., Sun, F., Wu, Y., Tian, Y., Vajda, P., Jia, Y., Keutzer, K., 2019. Fbnet: Hardware-aware efficient convnet design via differentiable neural architecture search, in: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR),...

    2019

  44. [52]

    Designing energy-efficient convolutional neural networks using energy-aware pruning, in: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp

    Yang, T.J., Chen, Y.H., Sze, V., 2017. Designing energy-efficient convolutional neural networks using energy-aware pruning, in: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 6071–6079. doi:10.1109/CVPR.2017.643

    2017 doi

  45. [53]

    A gift from knowledge distillation: Fast optimization, network minimization and transfer learning, in: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp

    Yim, J., Joo, D., Bae, J., Kim, J., 2017. A gift from knowledge distillation: Fast optimization, network minimization and transfer learning, in: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7130–7138. doi:10.1109/CVPR.2017.754

    2017 doi

  46. [54]

    Accelerating very deep convolutional networks for classification and detection

    Zhang, X., Zou, J., He, K., Sun, J., 2016. Accelerating very deep convolutional networks for classification and detection. IEEE Transactions on Pattern Analysis and Machine Intelligence 38, 1943–1955. doi:10.1109/TPAMI.2015.2502579

    2016

  47. [55]

    Efficientandaccurateapproximationsofnonlinearconvolutionalnetworks,in:2015IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp

    Zhang,X.,Zou,J.,Ming,X.,He,K.,Sun,J.,2015. Efficientandaccurateapproximationsofnonlinearconvolutionalnetworks,in:2015IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1984–1992. doi:10.1109/CVPR.2015.7298809

    2015

  48. [56]

    Object detection with deep learning: A review

    Zhao, Z.Q., Zheng, P., Xu, S.T., Wu, X., 2019. Object detection with deep learning: A review. IEEE Transactions on Neural Networks and Learning Systems 30, 3212–3232. doi:10.1109/TNNLS.2018.2876865

    2019

  49. [57]

    ProceedingsoftheAAAIConferenceonArtificialIntelligence32

    Zhou,G.,Fan,Y.,Cui,R.,Bian,W.,Zhu,X.,Gai,K.,2018.Rocketlaunching:Auniversalandefficientframeworkfortrainingwell-performing lightnet. ProceedingsoftheAAAIConferenceonArtificialIntelligence32. URL: https://ojs.aaai.org/index.php/AAAI/article/ view/11601, doi:10.1609/aaai.v32i1.11601

    2018 doi

  50. [58]

    Zhuang, Z., Tan, M., Zhuang, B., Liu, J., Guo, Y., Wu, Q., Huang, J., Zhu, J., 2018. Discrimination-aware channel pruning for deep neural networks,in:Proceedingsofthe32ndInternationalConferenceonNeuralInformationProcessingSystems,CurranAssociatesInc.,RedHook, NY, USA. p. 883–894

    2018

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.