REVIEW 17 cited by
Neural Architecture Search: Insights from 1000 Papers
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Neural Architecture Search: Insights from 1000 Papers
read the original abstract
In the past decade, advances in deep learning have resulted in breakthroughs in a variety of areas, including computer vision, natural language understanding, speech recognition, and reinforcement learning. Specialized, high-performing neural architectures are crucial to the success of deep learning in these areas. Neural architecture search (NAS), the process of automating the design of neural architectures for a given task, is an inevitable next step in automating machine learning and has already outpaced the best human-designed architectures on many tasks. In the past few years, research in NAS has been progressing rapidly, with over 1000 papers released since 2020 (Deng and Lindauer, 2021). In this survey, we provide an organized and comprehensive guide to neural architecture search. We give a taxonomy of search spaces, algorithms, and speedup techniques, and we discuss resources such as benchmarks, best practices, other surveys, and open-source libraries.
Forward citations
Cited by 17 Pith papers
-
Convergence Theory for Iterative LLM-Based Neural Architecture Search: A Parametric Cross-Entropy Framework with Closed-Form Proxy Reliability
Iterative LLM-NAS is equivalent to a parametric cross-entropy method with proven monotonic quality improvement, geometric convergence of elite probability, and a closed-form proxy reliability rho_S = (6/pi) arcsin(rho...
-
From Regression to Inference: Meta-Learning Predictors for Neural Architecture Search
Meta-learning a Convolutional Neural Process to infer neural architecture performance from context-target splits on synthesized tasks improves top-K ranking and achieves state-of-the-art selection on NAS-Bench-101 and...
-
Zero-Shot Neural Network Evaluation with Sample-Wise Activation Patterns
SWAP-Score evaluates neural networks without training by quantifying sample-wise activation patterns, achieving high correlation with true performance on CIFAR-10 for CNNs and GLUE for Transformers while enabling fast NAS.
-
Delta-Based Neural Architecture Search: LLM Fine-Tuning via Code Diffs
Fine-tuned 7B LLMs generating unified diffs for neural architecture refinement achieve 66-75% valid rates and 64-66% mean first-epoch accuracy, outperforming full-generation baselines by large margins while cutting ou...
-
Self-Directed Task Identification
SDTI lets models identify the correct target variable in datasets in a zero-shot setting using standard neural networks, beating baselines by 14% F1 on synthetic benchmarks.
-
Q-PhotoNAS: Hybrid Quantum Neural Architecture Search Framework on Photonic Devices
Q-PhotoNAS applies genetic algorithm search to jointly optimize classical preprocessing, phase encoding, and photonic circuit structure for hybrid quantum-classical models, reporting 99.44% and 98.78% accuracy on Digi...
-
OPT-BENCH: Evaluating the Iterative Self-Optimization of LLM Agents in Large-Scale Search Spaces
OPT-BENCH and OPT-Agent evaluate LLM self-optimization in large search spaces, showing stronger models improve via feedback but stay constrained by base capacity and below human performance.
-
Evolving Many Worlds: Towards Open-Ended Discovery in Petri Dish NCA via Population-Based Training
PBT-NCA evolves populations of Petri Dish NCAs under novelty-plus-diversity pressure, yielding sustained lifelike waves, spore-like colonization, and migrating macro-structures at the edge of chaos.
-
Evolving Many Worlds: Towards Open-Ended Discovery in Petri Dish NCA via Population-Based Training
PBT-NCA evolves PD-NCAs via a composite novelty-diversity objective to generate sustained emergent lifelike behaviors including waves, spore scattering, and migrating macro-structures at the edge of chaos.
-
Examining the Efficacy of Graph Neural Network Message-Passing in Regression Contexts
Across four NAS/DNN-predictor regression benchmarks, GEN (deep graph convolution) achieves the best average rank over 11 GNN message-passing layers, though attention GATv2 wins on the largest graphs.
-
Defending against Backdoor Attacks via Module Switching
Module-switching defense disrupts backdoors more effectively than weight averaging with fewer models and remains robust even when some models share the same backdoors.
-
On Efficient Variants of Segment Anything Model: A Survey
A survey that reviews efficient variants of the Segment Anything Model, categorizes acceleration strategies, and provides a unified hardware evaluation on benchmarks.
-
Running hardware-aware neural architecture search on embedded devices under 512MB of RAM
A HW-NAS framework executable on resource-limited embedded devices generates optimized CNNs for low-end MCUs and reports state-of-the-art human-recognition accuracy on the Visual Wake Word dataset.
-
Neural Architecture Search for Traffic Prediction: A Survey of Methods, Challenges, and Future Directions
NAS methods for traffic prediction are organized into three search-strategy families; cost-quality trade-offs and five open challenges are identified, with internal inconsistencies in the cost claims.
-
Universal Time-Series Representation Learning: A Survey
A survey that proposes a taxonomy for universal time-series representation learning and reviews existing deep learning studies along with experimental setups.
-
Neural Architecture Search for Generative Adversarial Networks: A Comprehensive Review and Critical Analysis
A literature review that categorizes and compares NAS techniques for GANs, noting benefits of evolutionary and gradient-based methods along with needs for better metrics and diverse datasets.
-
Spiking Neural Network Architecture Search: A Survey
A survey of Spiking Neural Network architecture search techniques viewed through a hardware/software co-design lens.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.