REVIEW 3 major objections 5 minor 47 references
HEP-NAS: Towards Efficient Few-shot Neural Architecture Search via Hierarchical Edge Partitioning
T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read By partitioning a NAS supernet around each intermediate node rather than edge by edge, and pruning to the best sub-supernet after each split, HEP-NAS achieves state-of-the-art few-shot search accuracy on CIFAR-10 (2.44% error), CIFAR-100…
desk verdict New hierarchy-wise splitting idea, strong empirical results, but the paper needs variance bars, an equal-cost ablation, and an oracle check on its greedy pruning before the efficiency claim is airtight. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The machinery has three parts. First, the hierarchy, defined as the set of all edges entering a given intermediate node, with operation groups on each edge obtained by gradient matching, so the Cartesian product of those groups creates the sub-supernets of that stage. Second, the search-space reduction step that keeps only the sub-supernet with the highest validation accuracy after a short warmup, pruning all other branches. Third, the search space mutual distillation loss, which adds to the classification loss a soft-target distillation term that pulls each sub-supernet toward both the previous optimal sub-supernet and its current siblings, stabilising training and accelerating convergence.
What would settle it
Compare final accuracy of HEP-NAS when the sub-supernet after each split is chosen by validation accuracy versus chosen at random with the same budget; if random choice nearly matches, the greedy selection is not carrying the benefit, and if validation-ranked selection wins, the early ranking is informative.
Extended reading notes
Core claim
The central claim is that partitioning the supernet by destination node, enumerating operation combinations on the edges entering each intermediate node instead of by individual edges, directly searches for the optimal operation combination per node and thereby mitigates co-adaptation more effectively. This hierarchy-wise partition creates a broad, shallow partition tree; after training the leaf sub-supernets with a distillation loss that combines the previous best sub-supernet and all current siblings, HEP-NAS selects the sub-supernet with the highest validation accuracy and discards the rest. Iterating over nodes in numbering order progressively narrows the search space while splitting all edges, and the authors demonstrate with Spearman correlation 0.665 that the final reduced space ranks architectures reliably. The paper claims this design is what allows it to outperform prior few-shot NAS methods on DARTS and NAS-Bench-201.
Load-bearing premise
The entire pipeline assumes that a sub-supernet's validation accuracy after only a few warmup epochs faithfully predicts which branch contains the best final architecture; if that early ranking is noisy, the pruning step deletes the optimal branch before it ever gets trained.
Editorial extensions
If this is right
- Few-shot NAS can split every edge in a large search space within a practical budget, because branch pruning keeps the number of trained sub-supernets small.
- Operation combinations at each intermediate node are compared head-to-head, so the selected architecture reflects node-level choices rather than isolated edge choices.
- Mutual distillation among sub-supernets and from the previous best sub-supernet makes early validation rankings informative enough to prune safely.
- The same pipeline transfers across datasets: searching on CIFAR-10 and reusing the cell gives 23.6% top-1 error when transferred to ImageNet.
Reading between the lines
- The hierarchy-wise partition can be applied to any cell-based or node-based search space; the key requirement is that edges sharing a destination node are grouped, which is true of DARTS, NAS-Bench-201, and most cell search spaces.
- The pruning step is a greedy algorithm; if early validation accuracy is a noisy proxy for final accuracy, the method could converge to a suboptimal local optimum. The paper reports a final Spearman correlation of 0.665 but does not measure the correlation at the warmup stage, which is the point where pruning decisions are made.
- A testable extension: replace the top-1 accuracy selection with an uncertainty-aware or ensemble-based selection to see if the method is less sensitive to warmup length.
- The SMD loss could be seen as a general stabiliser for sub-supernet training; combining it with other partition strategies, such as gradient-matching based grouping of operations within hierarchies, may further improve results.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes HEP-NAS, a few-shot neural architecture search method that partitions the supernet hierarchy-wise rather than edge-wise. Edges sharing the same end node are treated as one hierarchy; gradient matching first groups operations on each edge, and the Cartesian product of these groups over the hierarchy's incoming edges defines the sub-supernets for that hierarchy. After a short warm-up under a search-space mutual distillation objective (Eq. 7), only the sub-supernet with the highest validation accuracy is retained, and the process moves to the next hierarchy until all edges are split. The final architecture is read out from the remaining sub-supernet. Experiments on DARTS (CIFAR-10, CIFAR-100, ImageNet transfer) and NAS-Bench-201 (CIFAR-10, CIFAR-100, ImageNet16-120) report state-of-the-art or competitive accuracy with modest GPU-day costs.
Significance. If valid, the paper makes a useful contribution: it identifies a concrete limitation of edge-wise few-shot partitioning, namely persistent co-adaptation among edges that share an end node, and proposes a coarser-grained split that directly targets operation combinations at each intermediate node. The manuscript's strengths are the released code, the breadth of evaluation across two search spaces and several datasets, the ablation of the distillation components (Table 7), and the near-oracle performance on NAS-Bench-201 (e.g., 93.86% vs the optimal 94.37% test accuracy on CIFAR-10), which suggests that the pruning decisions do not, in practice, discard the globally best cell on that benchmark. The main claims would be materially strengthened by establishing that the early selection proxy is reliable and that the hierarchy-wise advantage is not an artifact of unequal search budgets.
major comments (3)
- [Search Space Reducing / Algorithm 1 (lines 8–11)] The greedy branch-selection step uses validation accuracy after only 5, 4, or 10 warm-up epochs (Eq. 5), and the paper itself states in the Search Space Reducing section that "too few epochs result in inaccurate performance evaluations." Because NAS-Bench-201 provides the true optimal architecture, the paper should report, for each hierarchy split and each run, whether the retained sub-supernet still contains the optimal cell, together with the rank correlation between the warm-up proxy and the oracle ranking. The currently reported Spearman value of 0.665 is measured only on the final reduced search space and has no baseline, so it does not establish that the early pruning decisions preserve the globally optimal architecture.
- [Experiments / Table 5 (ablation)] The hierarchy-wise versus edge-wise ablation uses different search costs (0.8 vs 0.36 GPU-days), so the reported improvements (2.51% vs 2.86% on CIFAR-10 and 17.08% vs 17.7% on CIFAR-100) could be attributable to the larger budget rather than to the splitting strategy. A budget-matched or iso-cost comparison is needed to support the claim that hierarchy-wise partitioning is the cause of the improvement. In addition, the hierarchy-wise cost of 0.8 GPU-days in Table 5 differs from the 1.5 and 1.6 GPU-days reported for HEP-NAS in Tables 1 and 2 under what appear to be the same experimental settings; this discrepancy should be explained.
- [Experiments / Tables 1–3] The text states that results are obtained in 4 independent runs, but Tables 1–3 report a single error rate without variance. Since the CIFAR-10 comparison with GM-DARTS is 2.44% versus 2.46%, the claimed superiority is not statistically meaningful without standard deviations or per-run values. Please add these, or explicitly state which quantities are averages and over how many runs they were computed.
minor comments (5)
- [Methodology, Eq. (6)] Equation (6) is a KL divergence, not the cross entropy with soft target labels described in the text; please correct the wording and state whether any temperature is applied to the soft targets.
- [Methodology, Eq. (1)] Please define the loss L and the shared weights ω in Eq. (1); currently ∇L(Mk_oi, ω) and the averaging implicit in Eq. (2) are underspecified.
- [Algorithm 1, line 9] Please clarify whether the "other sub-supernets" in Eq. (7) are evaluated with weights from the previous epoch or from the current epoch during the sequential training described in line 9.
- [Experiments / Table 6] The random-order row is averaged over 6 runs but no standard deviation is given; please add it so that the comparison with the 4-run results is meaningful.
- [Throughout] There are minor typographical issues, e.g., "comparation" in the caption of Fig. 2 and "correspond-ing" in Section Search Space Reducing; a careful proofread would improve presentation.
Circularity Check
No significant circularity; the pipeline is an algorithmic heuristic validated against external benchmarks, not a derivation that reduces to its own inputs.
full rationale
HEP-NAS's chain is algorithmic rather than definitional. The hierarchy-wise partition (Eqs. 3-4) enumerates operation-group combinations per intermediate node, and the selection step (Eq. 5, Algorithm 1 lines 8-11) is an explicit greedy validation heuristic, not a fitted parameter later renamed as a prediction. The reported test accuracies come from retraining the derived architecture on CIFAR-10/100/ImageNet and from the NAS-Bench-201 API oracle, so the headline results are externally grounded rather than forced by the selection criterion. SMD (Eq. 7) is a training objective with an ablation study, not an input that determines the final architecture by construction. Gradient matching (Eqs. 1-2) is cited from prior external work (Hu et al. 2022), not from the present authors, so no self-citation chain is load-bearing. The paper's own caution that 'too few epochs result in inaccurate performance evaluations due to delayed convergence' is a robustness concern about the greedy pruning proxy, not evidence that any equation is equivalent to another by construction. No uniqueness theorem is invoked, and no known result is merely renamed. The fragility of validating sub-supernets after only 5-10 warmup epochs is a legitimate correctness risk, but it is a heuristic limitation rather than circularity. Therefore the appropriate finding is no significant circularity.
Assumptions & free parameters
free parameters (3)
- split schedule (split epos) =
DARTS: 15, 25, 35, 45; NAS-Bench-201: 10, 20, 30
- warmup epochs per hierarchy (warmup epo) =
DARTS: 5 then 4; NAS-Bench-201: 10
- number of partitioned hierarchies =
DARTS: 4; NAS-Bench-201: 3
assumptions (3)
- domain assumption Gradient matching (Eq. 1) with cosine similarity on standalone gradients, plus min-cut grouping (Eq. 2), partitions operations so that any combination within a hierarchy is a meaningful candidate.
- domain assumption Validation accuracy after a few epochs of SMD training ranks branch quality reliably enough to prune.
- domain assumption Weights inherited from the parent supernet make sub-supernet evaluations comparable after splitting.
Cite this review
Pith. "Pith review of HEP-NAS: Towards Efficient Few-shot Neural Architecture Search via Hierarchical Edge Partitioning." pith.science (2026). https://pith.science/paper/22FACA3W
@misc{pith2026241210723,
author = {Pith},
title = {Pith review of: HEP-NAS: Towards Efficient Few-shot Neural Architecture Search via Hierarchical Edge Partitioning},
year = {2026},
howpublished = {\url{https://pith.science/paper/22FACA3W}},
note = {Machine review of arXiv:2412.10723}
}
read the original abstract
One-shot methods have significantly advanced the field of neural architecture search (NAS) by adopting weight-sharing strategy to reduce search costs. However, the accuracy of performance estimation can be compromised by co-adaptation. Few-shot methods divide the entire supernet into individual sub-supernets by splitting edge by edge to alleviate this issue, yet neglect relationships among edges and result in performance degradation on huge search space. In this paper, we introduce HEP-NAS, a hierarchy-wise partition algorithm designed to further enhance accuracy. To begin with, HEP-NAS treats edges sharing the same end node as a hierarchy, permuting and splitting edges within the same hierarchy to directly search for the optimal operation combination for each intermediate node. This approach aligns more closely with the ultimate goal of NAS. Furthermore, HEP-NAS selects the most promising sub-supernet after each segmentation, progressively narrowing the search space in which the optimal architecture may exist. To improve performance evaluation of sub-supernets, HEP-NAS employs search space mutual distillation, stabilizing the training process and accelerating the convergence of each individual sub-supernet. Within a given budget, HEP-NAS enables the splitting of all edges and gradually searches for architectures with higher accuracy. Experimental results across various datasets and search spaces demonstrate the superiority of HEP-NAS compared to state-of-the-art methods.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
, " * write output.state after.block = add.period write newline
ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...
-
[2]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
-
[3]
Bender, G.; Kindermans, P.-J.; Zoph, B.; Vasudevan, V.; and Le, Q. 2018. Understanding and simplifying one-shot architecture search. In International conference on machine learning, 550--559. PMLR
work page 2018
-
[4]
Cai, H.; Zhu, L.; and Han, S. 2018. Proxylessnas: Direct neural architecture search on target task and hardware. arXiv preprint arXiv:1812.00332
arXiv 2018
-
[5]
Cai, Z.; Chen, L.; Liu, P.; Ling, T.; and Lai, Y. 2024. EG-NAS: Neural Architecture Search with Fast Evolutionary Exploration. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, 11159--11167
work page 2024
-
[6]
Chen, X.; Xie, L.; Wu, J.; and Tian, Q. 2019. Progressive differentiable architecture search: Bridging the depth gap between search and evaluation. In Proceedings of the IEEE/CVF international conference on computer vision, 1294--1303
work page 2019
-
[7]
Chen, Z.; Qiu, G.; Li, P.; Zhu, L.; Yang, X.; and Sheng, B. 2023. Mngnas: distilling adaptive combination of multiple searched networks for one-shot neural architecture search. IEEE Transactions on Pattern Analysis and Machine Intelligence
work page 2023
-
[8]
Chu, X.; Lu, S.; Li, X.; and Zhang, B. 2023. Mixpath: A unified approach for one-shot neural architecture search. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 5972--5981
work page 2023
Show all 47 references
-
[9]
Chu, X.; Wang, X.; Zhang, B.; Lu, S.; Wei, X.; and Yan, J. 2020 a . Darts-: robustly stepping out of performance collapse without indicators. arXiv preprint arXiv:2009.01027
2020 arXiv
-
[10]
Chu, X.; Zhou, T.; Zhang, B.; and Li, J. 2020 b . Fair darts: Eliminating unfair advantages in differentiable architecture search. In European conference on computer vision, 465--480. Springer
2020
-
[11]
Dong, P.; Niu, X.; Li, L.; Tian, Z.; Wang, X.; Wei, Z.; Pan, H.; and Li, D. 2023. RD-NAS: Enhancing one-shot supernet ranking ability via ranking distillation from zero-cost proxies. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (...
2023
-
[12]
Dong, X.; and Yang, Y. 2019. Searching for a robust neural architecture in four gpu hours. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 1761--1770
2019
-
[13]
Dong, X.; and Yang, Y. 2020. Nas-bench-201: Extending the scope of reproducible neural architecture search. arXiv preprint arXiv:2001.00326
2020 arXiv
-
[14]
Guo, Z.; Zhang, X.; Mu, H.; Heng, W.; Liu, Z.; Wei, Y.; and Sun, J. 2020. Single path one-shot neural architecture search with uniform sampling. In Computer Vision--ECCV 2020: 16th European Conference, Glasgow, UK, August 23--28, 2020, Proceedings, Part XVI 16, 544--560. Springer
2020
-
[15]
He, H.; Liu, L.; Zhang, H.; and Zheng, N. 2024. IS-DARTS: Stabilizing DARTS through Precise Measurement on Candidate Importance. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, 12367--12375
2024
-
[16]
He, K.; Zhang, X.; Ren, S.; and Sun, J. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, 770--778
2016
-
[17]
Howard, A.; Sandler, M.; Chu, G.; Chen, L.-C.; Chen, B.; Tan, M.; Wang, W.; Zhu, Y.; Pang, R.; Vasudevan, V.; et al. 2019. Searching for mobilenetv3. In Proceedings of the IEEE/CVF international conference on computer vision, 1314--1324
2019
-
[18]
Hu, S.; Wang, R.; Hong, L.; Li, Z.; Hsieh, C.-J.; and Feng, J. 2022. Generalizing few-shot nas with gradient matching. arXiv preprint arXiv:2203.15207
2022 arXiv
-
[19]
Huang, G.; Liu, Z.; Van Der Maaten, L.; and Weinberger, K. Q. 2017. Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, 4700--4708
2017
-
[20]
Jiang, S.; Ji, Z.; Zhu, G.; Yuan, C.; and Huang, Y. 2024. Operation-level early stopping for robustifying differentiable NAS. Advances in Neural Information Processing Systems, 36
2024
-
[21]
Jiang, T.; Wang, H.; and Bie, R. 2024. MeCo: zero-shot NAS with one data and single forward pass via minimum eigenvalue of correlation. Advances in Neural Information Processing Systems, 36
2024
-
[22]
Li, G.; Yang, Y.; Bhardwaj, K.; and Marculescu, R. 2023. Zico: Zero-shot nas via inverse coefficient of variation on gradients. arXiv preprint arXiv:2301.11300
2023 arXiv
-
[23]
Li, L.; and Talwalkar, A. 2020. Random search and reproducibility for neural architecture search. In Uncertainty in artificial intelligence, 367--377. PMLR
2020
-
[24]
Lin, M.; Wang, P.; Sun, Z.; Chen, H.; Sun, X.; Qian, Q.; Li, H.; and Jin, R. 2021. Zen-nas: A zero-shot nas for high-performance image recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 347--356
2021
-
[25]
Liu, H.; Simonyan, K.; and Yang, Y. 2018. Darts: Differentiable architecture search. arXiv preprint arXiv:1806.09055
2018 arXiv
-
[26]
A.; Hacene, G
Ly-Manson, T.; Leonardon, M.; El Bey, A. A.; Hacene, G. B.; and Mauch, L. 2024. Analyzing Few-Shot Neural Architecture Search in a Metric-Driven Framework
2024
-
[27]
Ma, N.; Zhang, X.; Zheng, H.-T.; and Sun, J. 2018. Shufflenet v2: Practical guidelines for efficient cnn architecture design. In Proceedings of the European conference on computer vision (ECCV), 116--131
2018
-
[28]
Pham, H.; Guan, M.; Zoph, B.; Le, Q.; and Dean, J. 2018. Efficient neural architecture search via parameters sharing. In International conference on machine learning, 4095--4104. PMLR
2018
-
[29]
Real, E.; Aggarwal, A.; Huang, Y.; and Le, Q. V. 2019. Regularized evolution for image classifier architecture search. In Proceedings of the aaai conference on artificial intelligence, volume 33, 4780--4789
2019
-
[30]
M.; and Li, Z
Wan, X.; Ru, B.; Esperan c a, P. M.; and Li, Z. 2022. On redundancy and diversity in cell-based neural architecture search. arXiv preprint arXiv:2203.08887
2022 arXiv
-
[31]
Wang, R.; Cheng, M.; Chen, X.; Tang, X.; and Hsieh, C.-J. 2021. Rethinking architecture selection in differentiable NAS. arXiv preprint arXiv:2108.04392
2021 arXiv
-
[32]
Wang, W.; Zhang, X.; Cui, H.; Yin, H.; and Zhang, Y. 2023. FP-DARTS: Fast parallel differentiable neural architecture search for image classification. Pattern Recognition, 136: 109193
2023
-
[33]
Wu, F.; Gao, J.; Hong, L.; Wang, X.; Zhou, C.; and Ye, N. 2024. G-NAS: Generalizable Neural Architecture Search for Single Domain Generalization Object Detection. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, 5958--5966
2024
-
[34]
Xiao, H.; Wang, Z.; Zhu, Z.; Zhou, J.; and Lu, J. 2022. Shapley-NAS: discovering operation contribution for neural architecture search. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 11892--11901
2022
-
[35]
Xie, S.; Zheng, H.; Liu, C.; and Lin, L. 2018. SNAS: stochastic neural architecture search. arXiv preprint arXiv:1812.09926
2018 arXiv
-
[36]
Xu, Y.; Xie, L.; Zhang, X.; Chen, X.; Qi, G.-J.; Tian, Q.; and Xiong, H. 2019. Pc-darts: Partial channel connections for memory-efficient architecture search. arXiv preprint arXiv:1907.05737
2019 arXiv
-
[37]
Xue, Y.; Chen, C.; and Słowik, A. 2023. Neural Architecture Search Based on a Multi-Objective Evolutionary Algorithm With Probability Stack. IEEE Transactions on Evolutionary Computation, 27(4): 778--786
2023
-
[38]
Xue, Y.; and Qin, J. 2023. Partial Connection Based on Channel Attention for Differentiable Neural Architecture Search. IEEE Transactions on Industrial Informatics, 19(5): 6804--6813
2023
-
[39]
Yu, H.; Peng, H.; Huang, Y.; Fu, J.; Du, H.; Wang, L.; and Ling, H. 2022. Cyclic differentiable architecture search. IEEE transactions on pattern analysis and machine intelligence, 45(1): 211--228
2022
-
[40]
Yuan, G.; Wang, B.; Xue, B.; and Zhang, M. 2023. Particle swarm optimization for efficiently evolving deep convolutional neural networks using an autoencoder-based encoding strategy. IEEE Transactions on Evolutionary Computation
2023
-
[41]
Yuan, G.; Xue, B.; and Zhang, M. 2023. An effective one-shot neural architecture search method with supernet fine-tuning for image classification. In Proceedings of the Genetic and Evolutionary Computation Conference, 615--623
2023
-
[42]
Zhang, B.; Wang, X.; Qin, X.; and Yan, J. 2024. Boosting Order-Preserving and Transferability for Neural Architecture Search: a Joint Architecture Refined Search and Fine-tuning Approach. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 5662--5671
2024
-
[43]
Zhang, M.; Li, H.; Pan, S.; Chang, X.; and Su, S. 2020. Overcoming multi-model forgetting in one-shot NAS with diversity maximization. In Proceedings of the ieee/cvf conference on computer vision and pattern recognition, 7809--7818
2020
-
[44]
Zhang, X.; Zhou, X.; Lin, M.; and Sun, J. 2018. Shufflenet: An extremely efficient convolutional neural network for mobile devices. In Proceedings of the IEEE conference on computer vision and pattern recognition, 6848--6856
2018
-
[45]
Zhao, Y.; Wang, L.; Tian, Y.; Fonseca, R.; and Guo, T. 2021. Few-shot neural architecture search. In International Conference on Machine Learning, 12707--12718. PMLR
2021
-
[46]
Zheng, X.; Yang, C.; Zhang, S.; Wang, Y.; Zhang, B.; Wu, Y.; Wu, Y.; Shao, L.; and Ji, R. 2023. Ddpnas: Efficient neural architecture search via dynamic distribution pruning. International Journal of Computer Vision, 131(5): 1234--1249
2023
-
[47]
Zoph, B.; and Le, Q. V. 2016. Neural architecture search with reinforcement learning. arXiv preprint arXiv:1611.01578
2016 arXiv
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.