REVIEW 9 cited by
NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Neural architecture search (NAS) has achieved breakthrough success in a great number of applications in the past few years. It could be time to take a step back and analyze the good and bad aspects in the field of NAS. A variety of algorithms search architectures under different search space. These searched architectures are trained using different setups, e.g., hyper-parameters, data augmentation, regularization. This raises a comparability problem when comparing the performance of various NAS algorithms. NAS-Bench-101 has shown success to alleviate this problem. In this work, we propose an extension to NAS-Bench-101: NAS-Bench-201 with a different search space, results on multiple datasets, and more diagnostic information. NAS-Bench-201 has a fixed search space and provides a unified benchmark for almost any up-to-date NAS algorithms. The design of our search space is inspired from the one used in the most popular cell-based searching algorithms, where a cell is represented as a DAG. Each edge here is associated with an operation selected from a predefined operation set. For it to be applicable for all NAS algorithms, the search space defined in NAS-Bench-201 includes all possible architectures generated by 4 nodes and 5 associated operation options, which results in 15,625 candidates in total. The training log and the performance for each architecture candidate are provided for three datasets. This allows researchers to avoid unnecessary repetitive training for selected candidate and focus solely on the search algorithm itself. The training time saved for every candidate also largely improves the efficiency of many methods. We provide additional diagnostic information such as fine-grained loss and accuracy, which can give inspirations to new designs of NAS algorithms. In further support, we have analyzed it from many aspects and benchmarked 10 recent NAS algorithms.
Forward citations
Cited by 9 Pith papers
-
HamQASBench: A Hamiltonian-Informed Diagnostic Benchmark for Evaluating Quantum Architecture Search
A five-tier, Hamiltonian-fingerprint benchmark plus critical-structure extraction exposes QAS failure modes that energy-only metrics miss across eleven molecules.
-
NN-Former: Rethinking Graph Structure in Neural Architecture Representation
NN-Former improves neural accuracy and latency prediction by using attention masks over sibling nodes in the architecture graph.
-
Language Embedding Meets Dynamic Graph: A New Exploration for Neural Architecture Representation Learning
LeDG-Former improves neural architecture latency prediction by combining BERT-based language embeddings of architectures and hardware with dynamic graph self-attention, and reports SOTA on NNLQP.
-
Gradient Flow Matching for Learning Update Dynamics in Neural Network Training
GFM applies conditional flow matching to neural weight trajectories, predicting final weights from a short observed prefix with accuracy competitive to Transformers on synthetic and CIFAR-10 tasks.
-
HiFi-LLP: High-Fidelity, Low-Cost Latency Predictors with Confidence for Robust HW-NAS
A GATv2+GP latency predictor trained on 100 samples yields high rank fidelity and a confidence-aware hybrid HW-NAS that routes uncertain queries to hardware-in-the-loop.
-
Coflex: Enhancing HW-NAS with Sparse Gaussian Processes for Efficient and Scalable DNN Accelerator Design
Coflex applies sparse Gaussian processes to multi-objective hardware-aware NAS, claiming near-linear scaling and superior Pareto fronts for DNN accelerator co-design.
-
PhaseNAS: Language-Model Driven Architecture Search with Dynamic Phase Adaptation
PhaseNAS uses dynamic small-to-large LLM switching and a template language to search neural architectures, claiming better accuracy and lower search cost on NAS-Bench-Macro, CIFAR, and COCO.
-
Deep Electromagnetic Structure Design Under Limited Evaluation Budgets
A progressive quadtree search with consistency-based sample selection designs electromagnetic structures with 1000 simulations, beating baselines that use up to 7000.
-
Approach to Finding a Robust Deep Learning Model
A robustness measure based on the spread of test losses across independently trained instances, plus a pruning algorithm, selects stable small CNNs for calorimeter energy and position reconstruction.
Discussion (0). Continue with ORCID to comment.