A Comprehensive Survey on Hardware-Aware Neural Architecture Search
Reviewed by Pithpith:THZIFK53open to challenge →
read the original abstract
Neural Architecture Search (NAS) methods have been growing in popularity. These techniques have been fundamental to automate and speed up the time consuming and error-prone process of synthesizing novel Deep Learning (DL) architectures. NAS has been extensively studied in the past few years. Arguably their most significant impact has been in image classification and object detection tasks where the state of the art results have been obtained. Despite the significant success achieved to date, applying NAS to real-world problems still poses significant challenges and is not widely practical. In general, the synthesized Convolution Neural Network (CNN) architectures are too complex to be deployed in resource-limited platforms, such as IoT, mobile, and embedded systems. One solution growing in popularity is to use multi-objective optimization algorithms in the NAS search strategy by taking into account execution latency, energy consumption, memory footprint, etc. This kind of NAS, called hardware-aware NAS (HW-NAS), makes searching the most efficient architecture more complicated and opens several questions. In this survey, we provide a detailed review of existing HW-NAS research and categorize them according to four key dimensions: the search space, the search strategy, the acceleration technique, and the hardware cost estimation strategies. We further discuss the challenges and limitations of existing approaches and potential future directions. This is the first survey paper focusing on hardware-aware NAS. We hope it serves as a valuable reference for the various techniques and algorithms discussed and paves the road for future research towards hardware-aware NAS.
This paper has not been read by Pith yet.
Forward citations
Cited by 8 Pith papers
-
Protecting On-Device AI Inference: A Systematic Review of Attacks and Defence Mechanisms
A systematic review of on-device AI inference security finds defenses are imbalanced, with roughly half focused on IP theft while one-third of attacks (adversarial examples) lack any associated defenses.
-
Smaug: Fixing Failure Modes of Preference Optimisation with DPO-Positive
DPOP is a new loss function that prevents DPO from lowering preferred response likelihoods and outperforms standard DPO on diverse datasets, MT-Bench, and enables Smaug-72B to exceed 80% on the Open LLM Leaderboard.
-
Harvesting AI Computation at the Edge via Generic Approximation
A framework converts traditional edge tasks to NN models via NAS and schedules them on idle AI chips to improve performance without affecting primary workloads.
-
Bayesian Optimization of Crossbar-Based Compute-In-Memory System Design for Efficient DNN Inference
A multi-objective Bayesian optimization framework co-optimizes CIM crossbar hardware and DNN parameters for VGG8/CIFAR-10 and VGG16/Tiny-ImageNet, achieving comparable accuracy with up to 65% smaller area and 52% lowe...
-
Hybrid Quantum-Classical Neural Architecture Search
Demonstrates FLOPs-aware neural architecture search for hybrid quantum-classical neural networks to produce accurate yet computationally efficient models suitable for NISQ hardware.
-
Edge Deep Learning in Computer Vision and Medical Diagnostics: A Comprehensive Survey
A comprehensive survey of edge deep learning in computer vision and medical diagnostics that presents a novel categorization of hardware platforms by performance and usage scenarios.
-
elasticAI.explorer: Towards a Unified End-to-End Framework for Hardware-Aware Neural Architecture Search
elasticAI.explorer is an extensible framework for hardware-aware NAS supporting multiple search space types with YAML specs, code generation, cross-compilation, and on-device benchmarking.
-
Spiking Neural Network Architecture Search: A Survey
A survey of Spiking Neural Network architecture search techniques viewed through a hardware/software co-design lens.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.