REVIEW 16 cited by
Neural Architecture Search: Insights from 1000 Papers
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
In the past decade, advances in deep learning have resulted in breakthroughs in a variety of areas, including computer vision, natural language understanding, speech recognition, and reinforcement learning. Specialized, high-performing neural architectures are crucial to the success of deep learning in these areas. Neural architecture search (NAS), the process of automating the design of neural architectures for a given task, is an inevitable next step in automating machine learning and has already outpaced the best human-designed architectures on many tasks. In the past few years, research in NAS has been progressing rapidly, with over 1000 papers released since 2020 (Deng and Lindauer, 2021). In this survey, we provide an organized and comprehensive guide to neural architecture search. We give a taxonomy of search spaces, algorithms, and speedup techniques, and we discuss resources such as benchmarks, best practices, other surveys, and open-source libraries.
Forward citations
Cited by 16 Pith papers
-
Can Transformers Really Do It All? On the Compatibility of Inductive Biases Across Tasks
Learned replacement non-linearities show transformers are rarely optimal for algorithmic tasks, with benefits that are task-specific, while language/code gains are smaller and more transferable.
-
Self-Supervised Representation-Guided Generative Dataset Distillation
SRG guides diffusion-based dataset distillation with self-supervised representation prototypes, beating generative baselines for frozen pretrained encoders.
-
GraphIR: Architecture-Level Search States for LLM-Guided Neural Architecture Evolution
GraphIR adds a structured architecture state to executable neural programs for LLM-guided NAS; benchmark gains are reported, but the CLRS evidence shows the 'evolved' architecture is essentially the initial one plus a...
-
ST-LoRA: Single Trajectory LoRA Ensemble for Uncertainty Aware Agricultural Segmentation
Combining LoRA with snapshot ensembling yields a parameter-efficient uncertainty-aware segmentation ensemble that matches snapshot full-rank baselines, with feed-forward layers identified as the critical LoRA target.
-
Q-PhotoNAS: Hybrid Quantum Neural Architecture Search Framework on Photonic Devices
Q-PhotoNAS applies genetic algorithm search to jointly optimize classical preprocessing, phase encoding, and photonic circuit structure for hybrid quantum-classical models, reporting 99.44% and 98.78% accuracy on Digi...
-
Plug-in and Fine-tuning: Bridging the Gap between Small Language Models and Large Language Models
PiFi adds one frozen LLM layer to an SLM and fine-tunes it, reporting consistent but modest gains across NLU and NLG tasks, with larger gains when the LLM matches the target language.
-
Global optimization of graph acquisition functions for neural architecture search
A mixed-integer programming formulation globally optimizes graph Bayesian optimization acquisition functions for neural architecture search, with a proved graph encoding.
-
LEMUR: Learning to Align with Multi-Objective Reinforcement Learning from Preference Feedback
LEMUR jointly learns a separate reward model for each teacher's preferences and uses them to train a population of multi-objective policies, beating baselines that merge feedback into one reward.
-
Examining the Efficacy of Graph Neural Network Message-Passing in Regression Contexts
Across four NAS/DNN-predictor regression benchmarks, GEN (deep graph convolution) achieves the best average rank over 11 GNN message-passing layers, though attention GATv2 wins on the largest graphs.
-
Performance and Complexity Trade-off Optimization of Speech Models During Training
By turning each layer's width into a continuous, noise-smoothed parameter, the authors train speech models whose sizes shrink during training, reducing FLOPs and size by roughly 80–90% in their case studies.
-
Coflex: Enhancing HW-NAS with Sparse Gaussian Processes for Efficient and Scalable DNN Accelerator Design
Coflex applies sparse Gaussian processes to multi-objective hardware-aware NAS, claiming near-linear scaling and superior Pareto fronts for DNN accelerator co-design.
-
LCS: An AI-based Low-Complexity Scaler for Power-Efficient Super-Resolution of Game Content
An efficient neural upscaler trained on natively rendered game image pairs beats traditional upscalers on the LPIPS and NIQE perceptual metrics, at a fraction of the parameter count.
-
confopt: A Library for Implementation and Evaluation of Gradient-based One-Shot NAS Methods
A library and nine DARTS-derived benchmarks show that relative rankings of gradient-based one-shot NAS methods are unstable, making DARTS-only evaluation unreliable.
-
Tuning without Peeking: Provable Generalization Bounds and Robust LLM Post-Training
A black-box evolutionary post-training method for LLMs with compression-based generalization and robustness bounds, showing small empirical gains on math reasoning plus strong resistance to membership inference.
-
Energy-Efficient Deep Learning for Traffic Classification on Microcontrollers
A hardware-aware searched 1D-CNN classifies encrypted traffic on STM32 microcontrollers with 96.59% accuracy, 10.08M FLOPs, and 7.86-29.10 mJ per inference.
-
Neural Architecture Search for Traffic Prediction: A Survey of Methods, Challenges, and Future Directions
NAS methods for traffic prediction are organized into three search-strategy families; cost-quality trade-offs and five open challenges are identified, with internal inconsistencies in the cost claims.
Discussion (0). Continue with ORCID to comment.