REVIEW 4 major objections 4 minor 20 references
PhaseNAS: Language-Model Driven Architecture Search with Dynamic Phase Adaptation
T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read PhaseNAS claims that dynamically switching LLM sizes by score threshold finds better architectures with up to 86% less search time.
desk verdict Dynamic LLM-switching is a useful NAS idea and the classification results are credible, but the detection score is unvalidated and the YOLO gains are not attributable as submitted. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The mechanism that carries the argument is a phase-aware controller that escalates LLM capability in response to real-time architecture scores. The controller's inputs are the search space $S$, an initial architecture, thresholds $\gamma_{\mathrm{trans}}$ and $\gamma_{\mathrm{stop}}$, and pool size $K$; its output is the architecture with the highest zero-shot score. Two design choices make the controller effective: a parameterized template language—for example $\mathrm{ConvK3BNRELU}(3,8,1,1)$ or $\mathrm{ResK3K3}(16,32,2,1)$—that maps natural-language requests to compilable modules and reduces decoding failures, and task-specific scoring: the Zen-NAS score for classification, and a detection extension $\mathrm{NAS}_{\mathrm{det}} = \log(\Delta + \epsilon) + B$, where $\Delta$ is the summed $\ell^1$ perturbation response across feature scales and $B$ is a BatchNorm running-variance term, averaged over $R$ repetitions. The controller uses these scores both to rank candidates and to decide when to switch phases.
What would settle it
Compute $\mathrm{NAS}_{\mathrm{det}}$ for a varied set of, say, twenty YOLOv8 variants whose COCO mAP is already known, then measure the rank correlation between $\mathrm{NAS}_{\mathrm{det}}$ and mAP; a near-zero Spearman correlation would show the detection score does not guide the search toward better detectors, removing the basis for the YOLOv8 results.
Extended reading notes
Core claim
On its own terms, PhaseNAS claims that the main obstacle in LLM-based NAS is not the LLM's architectural knowledge but the allocation of its compute: broad exploration is wastefully expensive on a large model, and fine-grained refinement is too weak on a small one. The method therefore runs exploration with a smaller model, keeps a quality-ordered pool of at most K architectures, switches to a larger model once the best score reaches $\gamma_{\mathrm{trans}}$, and stops at $\gamma_{\mathrm{stop}}$. With this control, the paper reports finding a NAS-Bench-Macro architecture with 93.11% accuracy and rank 3, versus 92.73%/rank 119 and 92.75%/rank 110 for the same two models run independently as GUNIUS generators. On CIFAR-10/100 it reports matching or improving Zen-NAS accuracy while cutting search time by up to 86%, and on COCO it reports YOLOv8n* at 39.1 mAP and YOLOv8s* at 46.1 mAP, above the 37.3 and 44.9 baselines, with fewer parameters and FLOPs. The framework's broader claim is that the same phase-adaptive, template-structured procedure transfers across classification and detection tasks.
Load-bearing premise
The load-bearing premise is that the detection NAS score $\mathrm{NAS}_{\mathrm{det}} = \log(\Delta + \epsilon) + B$ correctly ranks candidate YOLO variants by their final COCO mAP; the paper neither reports a correlation analysis nor specifies the constants $\gamma$, $L$, $R$, and $\epsilon$, so if the score does not track mAP the reported detection gains cannot be attributed to PhaseNAS.
Editorial extensions
If this is right
- If the phase escalation rule is the source of the gains, then other LLM-driven generation loops beyond NAS can cut cost by matching model scale to the stage of the search.
- The template language indicates that a small set of structured primitives is enough for LLMs to produce valid, executable vision architectures, which should reduce wasted evaluations in future searches.
- A valid zero-shot detection score would extend NAS to object detectors without full training runs, allowing much larger detection search spaces to be explored under a fixed compute budget.
- The YOLOv8 results imply that LLM-guided search can improve a heavily hand-tuned detection model while also shrinking it, which bears directly on deployment in real-time systems.
Reading between the lines
- A testable extension of the paper is to run the same two-phase controller on a pure optimization problem, such as prompt or hyperparameter search, to see whether threshold-based escalation is a general cost-saving device rather than a NAS-specific trick.
- Because the constants $\gamma$, $L$, $R$, and $\epsilon$ in the detection score are unspecified, the score's ranking behavior should be checked against mAP before relying on it; this is a direct experiment the paper does not report.
- The NAS-Bench-Macro rank improvement from 119/110 to 3 could partly reflect the better exploration coverage of the smaller model in phase one; ablating the threshold value would show how sensitive the gain is to the switching rule.
- If the detection score is blind to mAP, the YOLOv8 gains might come from the restricted search space (YOLOv8+ versus YOLOv8*) rather than from adaptive search; searching the same space with a single large LLM would separate these two effects.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. PhaseNAS is an LLM-based neural architecture search framework that alternates between a small LLM for broad exploration and a larger LLM for focused refinement, with phase transitions and termination controlled by real-time score thresholds. Architectures are generated in a structured template language intended to reduce decoding failures. For classification, the framework reuses the Zen-Score; for object detection, it introduces a new score, NASdet, based on multi-scale feature perturbation responses and BatchNorm statistics. The paper reports results on NAS-Bench-Macro, CIFAR-10/100, and COCO with YOLOv8 variants, claiming better accuracy and rank than GUNIUS, up to 86% lower search time than Zen-NAS, and higher mAP with lower parameter and FLOP counts than the YOLOv8 baselines.
Significance. The dynamic phase-adaptation idea is a reasonable direction for LLM-based NAS, and extending zero-shot architecture scoring from classification to detection is a worthwhile research goal. The paper has several strengths: a clear algorithmic description (Algorithm 1), a comparison against an existing LLM-based NAS method on the established NAS-Bench-Macro benchmark, and the use of the already validated Zen-Score for the classification experiments. However, the detection results currently rest on an unvalidated scoring function and a search space that includes blocks from YOLOv10 known to improve performance independently of the search algorithm. Because the generalizability claim depends on the detection experiments, the central claim is not yet established as submitted.
major comments (4)
- [Section 2.5, Eq. (5)] The detection scoring function NASdet = log(Delta + epsilon) + B is introduced without any derivation, validation, or specified values for gamma, L, R, and epsilon. There is no correlation analysis between this score and final COCO mAP. Since Algorithm 1 uses this score for pool maintenance, phase transition, and termination, an uncalibrated score makes the detection search effectively blind. Please provide a derivation or empirical validation (e.g., Spearman correlation with trained mAP on a set of YOLO variants), state all constants and ranges, and include an ablation showing that search driven by NASdet outperforms random search in the same search space.
- [Section 3.4, Table 2] Table 2 reports only single runs of each YOLO variant and provides no comparison against another search algorithm over the same YOLOv8+ and YOLOv8* search spaces. Because the YOLOv8* space injects SCDown and PSA blocks from YOLOv10, which are known to improve detection performance independently of the search procedure, the reported mAP gains cannot be attributed to PhaseNAS. Add baselines such as random search and a fixed-LLM search over the same spaces, and report mean plus/minus standard deviation over at least three seeds.
- [Section 3.1, Fig. 2] The NAS-Bench-Macro comparison reports a single best accuracy and rank per method (92.73/119, 92.75/110, 93.11/3) without variance or significance testing, and the GUNIUS baseline is re-implemented with Qwen2.5 models instead of its original GPT-4 backbone. Please report the distribution of results over repeated runs and confirm that the adapted GUNIUS setup faithfully represents the baseline method.
- [Section 3.3, Table 1] The search-time comparison would be more convincing with a precise definition of what is measured (LLM inference time, architecture evaluation time, or wall-clock time including training), the hardware used, and the number of independent runs. The claimed up-to-86% reduction currently rests on single timing values, and the CIFAR-10 accuracy for PhaseNAS is sometimes lower than Zen-NAS, so the 'maintaining accuracy' claim needs statistical support.
minor comments (4)
- [Section 2.3, Algorithm 1] The symbol Ez(S) is used before the scoring functions are introduced in Section 2.5; please define Ez(S) in the algorithm caption as the task-specific score (Zen-Score for classification or NASdet for detection).
- [Section 2.2] The SCDown and PSA blocks are described as 'novel blocks' in the YOLOv8* search space; since these blocks come from YOLOv10, clarify that they are new to this search space rather than novel architectural contributions.
- [Table 1] The header 'CIF AR-10 Acc.' appears to contain a spacing or formatting error; please correct it and consider reporting accuracy with error bars.
- [Section 3.4] The training protocol for the YOLO variants (epochs, batch size, optimizer, augmentation, and hardware) is not reported, which makes the mAP comparison difficult to reproduce.
Circularity Check
No significant circularity: the paper's claimed predictions are benchmark-measured and do not reduce to their own inputs by construction.
full rationale
PhaseNAS's central results are not circular in the derivation sense. The NAS-Bench-Macro results are obtained by running the search and then reading measured accuracy and rank from the external benchmark; they are not reconstructed from the search score. The classification comparison uses the established Zen-Score as a search proxy while reporting independently measured CIFAR-10/100 accuracy, so the reported accuracies are not forced by the proxy. For detection, the NASdet score in Eq. (5) is an unvalidated heuristic, and the YOLOv8* search space includes SCDown and PSA blocks from YOLOv10, which is a genuine experimental confound; however, this is a correctness and attribution concern, not circularity. The paper does not fit any parameter to the final mAP and then rename it a prediction, and it does not rely on load-bearing self-citations. The unstated constants gamma, L, R, and epsilon impair reproducibility but do not make the derivation equivalent to its inputs. Therefore the paper receives a circularity score of 0.
Assumptions & free parameters
free parameters (7)
- gamma_trans (transition threshold)
- gamma_stop (stopping threshold)
- K (pool size)
- gamma (perturbation weight, Eq. 1)
- L (number of feature scales)
- R (number of score repetitions)
- epsilon (Eq. 5)
assumptions (4)
- domain assumption The detection score NASdet in Eq. 5 correlates with final mAP on COCO.
- domain assumption Zen-NAS score is a valid proxy for CIFAR accuracy.
- ad hoc to paper Smaller LLMs are sufficient for exploration and larger LLMs for refinement, and dynamic switching improves over fixed LLMs.
- domain assumption The structured template language reduces LLM generation errors and yields valid architectures.
Cite this review
Pith. "Pith review of PhaseNAS: Language-Model Driven Architecture Search with Dynamic Phase Adaptation." pith.science (2026). https://pith.science/paper/MPKQTPZD
@misc{pith2026250720592,
author = {Pith},
title = {Pith review of: PhaseNAS: Language-Model Driven Architecture Search with Dynamic Phase Adaptation},
year = {2026},
howpublished = {\url{https://pith.science/paper/MPKQTPZD}},
note = {Machine review of arXiv:2507.20592}
}
read the original abstract
Neural Architecture Search (NAS) is challenged by the trade-off between search space exploration and efficiency, especially for complex tasks. While recent LLM-based NAS methods have shown promise, they often suffer from static search strategies and ambiguous architecture representations. We propose PhaseNAS, an LLM-based NAS framework with dynamic phase transitions guided by real-time score thresholds and a structured architecture template language for consistent code generation. On the NAS-Bench-Macro benchmark, PhaseNAS consistently discovers architectures with higher accuracy and better rank. For image classification (CIFAR-10/100), PhaseNAS reduces search time by up to 86% while maintaining or improving accuracy. In object detection, it automatically produces YOLOv8 variants with higher mAP and lower resource cost. These results demonstrate that PhaseNAS enables efficient, adaptive, and generalizable NAS across diverse vision tasks.
Figures
Reference graph
Works this paper leans on
-
[1]
Journal of Machine Learning Research, 20(55), 1–21 (2019)
Elsken, T., Metzen, J.H., Hutter, F.: Neural architecture search: A survey. Journal of Machine Learning Research, 20(55), 1–21 (2019)
work page 2019
-
[2]
International Conference on Machine Learning
Ying, Chris, et al.: NAS-Bench-101: Towards reproducible neural architecture search. International Conference on Machine Learning. PMLR, 2019
work page 2019
-
[3]
In: Proceedings of the AAAI Conference on Artificial Intelligence, vol
Real, E., Aggarwal, A., Huang, Y., Le, Q.V.: Regularized evolution for image clas- sifier architecture search. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 4780–4789 (2019)
work page 2019
-
[4]
In: International Conference on Machine Learning, pp
Real, E., Moore, S., Selle, A., Saxena, S., Suematsu, Y.L., Tan, J., Le, Q.V., Ku- rakin, A.: Large-scale evolution of image classifiers. In: International Conference on Machine Learning, pp. 2902–2911 (2017)
work page 2017
-
[5]
arXiv preprint arXiv:1611.01578 (2016)
Zoph, B., Le, Q.: Neural architecture search with reinforcement learning. arXiv preprint arXiv:1611.01578 (2016)
arXiv 2016
-
[6]
In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp
Lin, M., Wang, P., Sun, Z., Chen, H., Sun, X., Qian, Q., Li, H., Jin, R.: Zen-NAS: A zero-shot NAS for high-performance image recognition. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 347–356 (2021)
work page 2021
-
[7]
In: International Conference on Learning Representations (2019)
Liu, H., Simonyan, K., Yang, Y.: DARTS: Differentiable architecture search. In: International Conference on Learning Representations (2019)
work page 2019
-
[8]
arXiv preprint arXiv:1905.01392 (2019)
Wistuba, M., Rawat, A., Pedapati, T.: A survey on neural architecture search. arXiv preprint arXiv:1905.01392 (2019)
arXiv 2019
Show all 20 references
-
[9]
Zheng, M., Su, X., You, S., Wang, F., Qian, C., Xu, C., Albanie, S.: Can GPT-4 perform neural architecture search? arXiv preprint arXiv:2304.10970 (2023)
2023 arXiv
-
[10]
Proceedings of the Genetic and Evolutionary Computation Conference (2024)
Nasir, Muhammad Umair, et al.: LLMatic: Neural architecture search via large language models and quality diversity optimization. Proceedings of the Genetic and Evolutionary Computation Conference (2024)
2024
-
[11]
Lemo-nade: Multi-parameter neural architecture discovery with LLMs
Rahman, Md Hafizur, and Prabuddha Chakraborty. Lemo-nade: Multi-parameter neural architecture discovery with LLMs. arXiv preprint arXiv:2402.18443 (2024)
2024 arXiv
-
[12]
arXiv preprint arXiv:2302.14838 (2023)
Chen, A., Dohan, D., So, D.R.: EvoPrompting: Language models for code-level neural architecture search. arXiv preprint arXiv:2302.14838 (2023)
2023 arXiv
-
[13]
arXiv preprint arXiv:2309.08532 (2023)
Guo, Q., Wang, R., Guo, J., Li, B., Song, K., Tan, X., Liu, G., Bian, J., Yang, Y.: Connecting large language models with evolutionary algorithms yields powerful prompt optimizers. arXiv preprint arXiv:2309.08532 (2023)
2023 arXiv
-
[14]
arXiv preprint arXiv:2401.10034 (2024) 14 Fei Kong et al
Wu, X., Wu, S.H., Wu, J., Feng, L., Tan, K.C.: Evolutionary computation in the era of large language model: Survey and roadmap. arXiv preprint arXiv:2401.10034 (2024) 14 Fei Kong et al
2024 arXiv
-
[15]
In: Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp
Tan, M., Le, Q.: EfficientDet: Scalable and efficient object detection. In: Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10781–10790 (2020)
2020
-
[16]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp
Li, C., Li, Y., Barbu, A., Rehg, J.: NAS-FPN: Learning scalable feature pyramid architecture for object detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7036–7045 (2020)
2020
-
[17]
GitHub repository
Jocher, G., Chaurasia, A., Qiu, J.: Ultralytics YOLOv8 (Version 8.0.0). GitHub repository. https://github.com/ultralytics/ultralytics (2023)
2023
-
[18]
arXiv preprint arXiv:2108.07732 (2021)
Austin, J., Odena, A., Nye, M., Bosma, M., Michalewski, H., Dohan, D., Jiang, E., Cai, C., Terry, M., Le, Q., et al.: Program synthesis with large language models. arXiv preprint arXiv:2108.07732 (2021)
2021 arXiv
-
[19]
arXiv preprint arXiv:2405.14458 (2024)
Wang, A., Chen, H., Liu, L., Chen, K., Lin, Z., Han, J., Ding, G.: YOLOv10: Real-Time End-to-End Object Detection. arXiv preprint arXiv:2405.14458 (2024)
2024 arXiv
-
[20]
NAS-Bench-201: Extending the scope of reproducible neural architecture search
Dong, Xuanyi, and Yi Yang. NAS-Bench-201: Extending the scope of reproducible neural architecture search. arXiv preprint arXiv:2001.00326 (2020)
2020 arXiv
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.