Pith. sign in

REVIEW 3 major objections 4 minor 45 references

Loss Functions for Predictor-based Neural Architecture Search

T0 review · 3 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Weighted loss functions best identify top-performing architectures in predictor-based NAS, and a piecewise ranking-to-weighted schedule (PWLNAS) outperforms every individual loss and prior methods.

desk verdict Broad, useful empirical comparison of loss functions for predictor-based NAS; the main trends hold up, but the PWLNAS recipe is per-task hand-picked on the same benchmarks, so the headline advantage is not yet a predictive claim. read the letter →

arxiv 2506.05869 v1 pith:CZGWT566 submitted 2025-06-06 cs.LG cs.AIcs.CV

Han Ji , Yuqi Feng , Jiahao Fan , Yanan Sun This is my paper
classification cs.LGcs.AIcs.CV
keywords neuralarchitecturesearchperformancepredictorslossfunctionsrankinglossesweightedpiecewiseNASbenchmarks
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks which loss function a performance predictor should use when estimating neural architecture quality, and it gives an answer grounded in experiments across 13 tasks and five search spaces. It categorizes losses into regression, ranking (pairwise and listwise), and weighted losses, and shows that no single type wins everywhere. Weighted losses are best at spotting the truly top architectures when enough training data are available, while ranking losses dominate when data are extremely scarce. The paper's main constructive claim is that these strengths can be combined: a piecewise loss that first trains the predictor with a ranking loss and then switches to a weighted loss (PWLNAS) beats every individual loss and several prior methods, reaching 2.47% test error on DARTS and 5.80% on NAS-Bench-101. If correct, the practical upshot is that loss choice should be guided by data budget and predictor backbone rather than treated as a fixed default.

What carries the argument

The central object is the piecewise (PW) loss schedule: the predictor is first trained with a ranking or regression loss to establish a global ordering of architectures, then switched to a weighted loss that concentrates gradient signal on high-accuracy candidates. The paper defines the loss landscape by dividing eight losses into regression, pairwise ranking, listwise ranking, and weighted categories, and evaluates them with Kendall's tau, Precision@T, and N@K metrics; the complementary behavior observed across these metrics is what motivates the PW design. PWLNAS is simply a predictor-based NAS method whose only change from a standard predictor is this PW loss schedule.

What would settle it

A controlled replication that fixes an equal hyperparameter-tuning budget per loss and chooses the piecewise switch point on a validation split or by a pre-specified rule would settle the central claim: if the piecewise loss then fails to beat the best single loss on the same benchmarks, the reported advantage is a tuning artifact rather than a property of the loss schedule.

Watch

Extended reading notes

Core claim

The paper establishes that loss functions in predictor-based NAS have complementary regimes. Weighted loss functions, including MAPE, EW, and WARP, achieve the best Precision@0.5 and N@10 on all tested search spaces, meaning they identify well-performing architectures, while ranking losses such as HR and ListMLE produce better overall ranking (Kendall's tau) and excel when training data are extremely limited, such as 0.1% of the search space. It also finds that simple MLP predictors work better with ranking losses, while GCN- and Transformer-based predictors benefit from weighted losses. On the basis of these regimes, the paper proposes PWLNAS, which trains the predictor with a regression or ranking loss in early iterations as a warm-up and switches to a weighted loss afterward. Across NAS-Bench-201, NAS-Bench-101, TransNAS-Bench-101 Micro, and DARTS, this piecewise loss outperforms every single loss and previous predictor-based NAS methods, with 2.47% test error on DARTS and 5.80% on NAS-Bench-101.

Load-bearing premise

The paper's loss rankings assume each loss was tuned with comparable effort and budget, a protocol the paper only mentions and defers to the supplementary material; it also sets PWLNAS's switch point using the same benchmarks on which it is evaluated.

Editorial extensions

If this is right

  • With very few labelled architectures, ranking losses such as HR and ListMLE are the safer choice: weighted losses overemphasize locally good samples and miss top architectures elsewhere in the search space.
  • With larger training sets, weighted losses (WARP, MAPE, EW) are the best tools for top-K identification, and their weights should use ground-truth accuracy rather than ranks.
  • Predictor backbone changes the best loss: MLP-based predictors pair better with ranking losses, while GCN- and Transformer-based predictors benefit more from weighted losses.
  • A piecewise schedule that warms up with ranking or regression and then switches to a weighted loss beats every single loss and previous methods, with 2.47% test error on DARTS and 5.80% on NAS-Bench-101.
  • More training data does not guarantee better top-ranking ability under random sampling, so sampling strategy is part of predictor effectiveness.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension the authors do not pursue is making the switch point adaptive: if the complementary regimes are real, a schedule that increases the weighting intensity as training data grows should match or beat the hand-set threshold, and this is directly testable on the same benchmarks.
  • The backbone-dependence result implies loss choice and predictor architecture should be co-designed; a concrete test is whether an MLP predictor with a ranking loss can match a GCN predictor with a weighted loss at equal query budgets.
  • The observation that larger random samples can hurt top-ranking ability suggests active sampling of diverse architectures would combine cleanly with weighted losses, and could make top-K precision monotonically improve with data.
  • Because the paper imports WARP from another application domain, the same weighting idea could transfer to other surrogate-based settings, such as predicting hardware latency or zero-cost proxies, where top-K identification matters more than absolute accuracy.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. This paper studies loss functions for predictor-based neural architecture search (NAS). It categorizes eight losses into regression (MSE), pairwise ranking (HR, LR, MSE+SR), listwise ranking (ListMLE), and weighted (EW, MAPE, WARP) families, and evaluates them on 13 tasks across five search spaces using Kendall's Tau, Precision@T, and N@K, with a uniform GCN-based predictor and variations in training portion and predictor backbone. The main empirical findings are that weighted losses are better at identifying top-performing architectures when training data are sufficient, ranking losses are better when data are extremely scarce, no single loss dominates across settings, and more training data can hurt top-K ability. Based on these trends, the authors propose PWLNAS, a predictor-based NAS method that uses a piecewise loss schedule (ranking/regression warm-up followed by a weighted loss), and report lower test error than single losses and prior state-of-the-art methods on NAS-Bench-201, NAS-Bench-101, TransNAS-Bench-101 Micro, and DARTS.

Significance. The descriptive part of the paper is a valuable systematization: it is among the broadest empirical comparisons of loss functions for NAS predictors, and the main qualitative trends are plausible, consistent across the reported spaces, and honestly presented, including the non-monotonic effect of training data on top-K ability. The proposed PWLNAS method, however, is not currently supported as a fixed, reusable algorithm because the loss composition and switch thresholds are chosen per task after observing the same benchmark results on which PWLNAS is then evaluated; the headline SOTA claims are therefore not yet established. If the authors add a prespecified, out-of-sample rule for choosing the piecewise recipe, document the per-loss tuning protocol, and provide the missing supplementary material and code, the paper would make a solid and useful contribution.

major comments (3)
  1. [Sec. 4.3, Sec. 5, Tables 2-5] The PWLNAS recipe is selected post hoc on the evaluation benchmarks. Section 4.3 states that "the choice of loss functions and the number of warm-up iterations depends on the specific task," and Section 5 concedes that "the value of the threshold to use different losses is fixed and relies on human experience." The specific compositions (HR+MAPE on NAS-Bench-201 and DARTS, ListMLE+WARP on NAS-Bench-101, MSE+EW on the Jigsaw task, HR+WARP on other TransNAS tasks) are chosen after the single-loss curves in Figures 2-3 and the searched results on those same spaces were observed. Under this protocol, a PW combination beating every single loss is expected even if the constituent losses are equivalent, because the pair and switch point are selected with access to the outcomes. To establish PWLNAS as an algorithm, the authors should fix a recipe-selection rule that does not use the evaluation results: for example, a rule based only on training portion and predictor backbone, evaluated on held-out tasks or search spaces, or a validation-based selection performed inside each run. Without such an out-of-sample test, the claim that PW loss outperforms every single loss is a test-set selection result rather than an evaluation of a predictive procedure.
  2. [Sec. 4.1 and multiple references to the Supplementary Material] The per-loss hyperparameter tuning protocol is not specified. Section 4.1 only says that "we use different levels of hyperparameters for each loss function" and defers details to the Supplementary Material, which is absent from this arXiv version; the same applies to the loss-function definitions, the PWLNAS pseudo-code, and the full per-task results. Without a documented tuning budget, search grid, selection criterion, and number of tuning trials per loss, cross-loss rankings such as "WARP outperforms HR in Precision@0.5 by 2.85%" (Section 4.2, Figure 5) and the backbone comparison in Table 1 could be artifacts of unequal tuning effort. The authors should include the supplementary material in the submission, state the tuning protocol precisely, and release code so that the comparison can be audited.
  3. [Sec. 4.3, Tables 2-5] The claimed state-of-the-art improvements are not statistically supported. On DARTS (Table 5), PWLNAS reports 2.47 +/- 0.05 versus DCLP's 2.48 +/- 0.02; the intervals overlap and no significance test is reported. Tables 2-4 report no variance or error bars even though Section 4.3 states that results are averaged over multiple runs (5 for DARTS, 20 for the others). The NAS-Bench-101 gain over NPENAS (5.80 vs 5.85) and the NAS-Bench-201 gains are also small relative to likely run-to-run variability. The authors should report per-run distributions, confidence intervals, or paired significance tests, and temper the "outperforms prior SOTA" wording unless the differences survive such analysis.
minor comments (4)
  1. [Sec. 4.2, Figure 6] The three weighting types are written as ‘GT’: \hat{y}, ‘EXP-GT’: exp(\hat{y}), and ‘Ranking’: 1-(r-1)/n, but \hat{y} was defined as the predicted score in Section 3; if these weights are supposed to use ground-truth performance, the notation should use y rather than \hat{y}.
  2. [Section 2 heading] The heading "Relate Works" should be "Related Work."
  3. [Sec. 4.2, Table 1 discussion] The explanation that WARP performs poorly with the MLP backbone "can be credited to its MLP backbone, which can easily overfit the limited training data and fall into the local optimal with the weighted loss function" is speculative; no diagnostics such as training curves or rank-correlation statistics are provided, so it should be phrased as a hypothesis or supported with evidence.
  4. [Sec. 4.2, mutation-based test] The mutation-based evaluation is conducted only on NAS-Bench-201, so the sentence "these results indicate that our findings still hold for neighborhood-based NAS methods" generalizes beyond the reported evidence; a caveat or additional spaces would be appropriate.

Circularity Check

1 steps flagged · score 6.0 of 10

PWLNAS's per-task piecewise-loss recipe is selected after seeing the same benchmark results on which it is evaluated, making its 'beats every single loss' headline a selection result rather than an out-of-sample prediction.

  1. fitted input called prediction [Section 4.3 (Predictor-based NAS evaluation) and Section 5 (Discussions and Suggestions)]
    "The choice of loss functions and the number of warm-up iterations depends on the specific task. ... Although the proposed PW loss is simple and effective, the value of the threshold to use different losses is fixed and relies on human experience."

    PWLNAS is not a fixed algorithm: the loss pair and warm-up length 'depend on the specific task', and the switch threshold 'relies on human experience'. These choices were made after seeing the same benchmark curves and search results on which PWLNAS is then evaluated (HR+MAPE for NB201/DARTS, ListMLE+WARP for NB101, MSE+EW for Jigsaw, HR+WARP elsewhere). Thus 'PW beats every single loss' (e.g. Table 3: PW 5.80 vs HR 5.83/ListMLE 5.84/WARP 5.86) is a post-hoc selection result: with freedom to pick the best pair after observing outcomes, outperforming every single loss is expected by construction even if the losses were equivalent. The missing Supplementary Material prevents auditing which switches were tried.

full rationale

The paper's benchmark study of eight loss functions is largely self-contained and not circular: Figures 2-5 report direct measurements on fixed GCN predictors and standard metrics, and no load-bearing self-citation is involved. The circularity is confined to the PWLNAS superiority claim. The PW recipe (which losses to combine and when to switch) is chosen per task after inspecting the same benchmark results used for evaluation, and Section 5 concedes the threshold is fixed by human experience. Consequently, the claim that PW beats every single loss is a test-set selection artifact, not an evaluation of a predictive procedure. The per-loss hyperparameter tuning protocol is also deferred to a missing Supplementary Material (Section 4.1), so cross-loss rankings could reflect tuning choices. These issues do not invalidate the raw measurements, but they do undermine the headline claim that PWLNAS is a method that outperforms prior SOTA as a fixed algorithm.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claims rest on four domain assumptions plus three free choices. No invented entities are introduced: PWLNAS is a training recipe, not a new object. The largest free parameters are the per-task PW loss pair and switch threshold, which the paper acknowledges are set by human experience on the same benchmarks used for evaluation.

free parameters (3)
  • PW loss switch threshold / warm-up iteration count = task-specific; early iteration count not specified in main text
    Chosen by hand per task based on observed behavior; Section 5 concedes the threshold is fixed and relies on human experience.
  • Per-loss hyperparameters (learning rate, etc.) = not reported in main text; deferred to Supplementary Material
    Different levels are used for each loss to make the comparison 'fair'; without a stated tuning budget, cross-loss rankings can be affected by tuning effort.
  • Choice of PW loss pair per task = HR+MAPE (NB201/DARTS), ListMLE+WARP (NB101), MSE+EW (Jigsaw), HR+WARP (other TransNAS tasks)
    Selected from the observed rankings of the same benchmark spaces on which PWLNAS is then compared with prior methods.
assumptions (4)
  • domain assumption A uniform GCN-based predictor is a representative backbone for predictor-based NAS
    Section 4.1 chooses GCN 'for its success in prior works'; most conclusions are drawn from this backbone, with only a single additional check on MLP and Transformer predictors.
  • domain assumption Validation accuracy trains the predictor and test accuracy evaluates it
    Section 4.2 follows prior works [21, 22]; if validation and test accuracy diverge across architectures, the ranking conclusions could shift.
  • domain assumption Excluding auxiliary losses (unsupervised, consistency) from scope is valid for comparing core losses
    Section 2 declares them out of scope; real predictors often use them, so the comparison is restricted to one design axis.
  • domain assumption Small random training subsets (0.1% to 3%) suffice to draw conclusions about full-space rankings
    Section 4.2 justifies small portions as practically relevant; the paper itself notes the noise this creates (top architectures appearing by chance in tiny subsets), mitigated by 30-run averaging without reported variance.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Loss Functions for Predictor-based Neural Architecture Search." pith.science (2026). https://pith.science/paper/CZGWT566

@misc{pith2026250605869,
  author       = {Pith},
  title        = {Pith review of: Loss Functions for Predictor-based Neural Architecture Search},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CZGWT566}},
  note         = {Machine review of arXiv:2506.05869}
}
read the original abstract

Evaluation is a critical but costly procedure in neural architecture search (NAS). Performance predictors have been widely adopted to reduce evaluation costs by directly estimating architecture performance. The effectiveness of predictors is heavily influenced by the choice of loss functions. While traditional predictors employ regression loss functions to evaluate the absolute accuracy of architectures, recent approaches have explored various ranking-based loss functions, such as pairwise and listwise ranking losses, to focus on the ranking of architecture performance. Despite their success in NAS, the effectiveness and characteristics of these loss functions have not been thoroughly investigated. In this paper, we conduct the first comprehensive study on loss functions in performance predictors, categorizing them into three main types: regression, ranking, and weighted loss functions. Specifically, we assess eight loss functions using a range of NAS-relevant metrics on 13 tasks across five search spaces. Our results reveal that specific categories of loss functions can be effectively combined to enhance predictor-based NAS. Furthermore, our findings could provide practical guidance for selecting appropriate loss functions for various tasks. We hope this work provides meaningful insights to guide the development of loss functions for predictor-based methods in the NAS community.

Figures

Figures reproduced from arXiv: 2506.05869 by the authors.

Figure 1
Figure 1. Illustration of main loss functions discussed in this paper. Ranking loss functions are further classified into pairwise and [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Precision@0.5, N@10, and Kendall’s Tau ( [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Precision@0.5, N@10, and Kendall’s Tau ( [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Precision@5 and N@10 of different loss functions for [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 6
Figure 6. Figure 6: Precision@T and N@K of different weighting types for [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Comparisons between architectures searched by different types of loss functions on NAS-Bench-201. [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

45 extracted references · 42 canonical work pages

  1. [1]

    Zero-cost proxies for lightweight nas.arXiv preprint arXiv:2101.08134, 2021

    Mohamed S Abdelfattah, Abhinav Mehrotra, Łukasz Dudziak, and Nicholas D Lane. Zero-cost proxies for lightweight nas.arXiv preprint arXiv:2101.08134, 2021. 1

    arXiv 2021

  2. [2]

    Once-for-all: Train one network and specialize it for efficient deployment.arXiv preprint arXiv:1908.09791,

    Han Cai, Chuang Gan, Tianzhe Wang, Zhekai Zhang, and Song Han. Once-for-all: Train one network and specialize it for efficient deployment.arXiv preprint arXiv:1908.09791,

    arXiv 1908

  3. [3]

    Nas-bench-201: Extending the scope of reproducible neural architecture search

    Xuanyi Dong and Yi Yang. Nas-bench-201: Extending the scope of reproducible neural architecture search. InProc. of ICLR, 2019. 2, 3, 4

    work page 2019

  4. [4]

    Transnas-bench-101: Improving transferability and generalizability of cross-task neural architecture search

    Yawen Duan, Xin Chen, Hang Xu, Zewei Chen, Xiaodan Liang, Tong Zhang, and Zhenguo Li. Transnas-bench-101: Improving transferability and generalizability of cross-task neural architecture search. InProc. of CVPR, 2021. 2, 3, 4

    work page 2021

  5. [5]

    Brp-nas: Prediction-based nas using gcns.Proc

    Lukasz Dudziak, Thomas Chau, Mohamed Abdelfattah, Royson Lee, Hyeji Kim, and Nicholas Lane. Brp-nas: Prediction-based nas using gcns.Proc. of NeurIPS, 2020. 4

    work page 2020

  6. [6]

    Neural architecture search: A survey.The Journal of Ma- chine Learning Research, 20(1):1997–2017, 2019

    Thomas Elsken, Jan Hendrik Metzen, and Frank Hutter. Neural architecture search: A survey.The Journal of Ma- chine Learning Research, 20(1):1997–2017, 2019. 1

    work page 1997

  7. [7]

    Bohb: Ro- bust and efficient hyperparameter optimization at scale

    Stefan Falkner, Aaron Klein, and Frank Hutter. Bohb: Ro- bust and efficient hyperparameter optimization at scale. In Proc. of ICML, 2018. 1

    work page 2018

  8. [8]

    Nas-fpn: Learning scalable feature pyramid architecture for object de- tection

    Golnaz Ghiasi, Tsung-Yi Lin, and Quoc V Le. Nas-fpn: Learning scalable feature pyramid architecture for object de- tection. InProc. of CVPR, 2019. 1

    work page 2019

Show all 45 references
  1. [9]

    Arch-graph: Acyclic architecture relation predictor for task-transferable neural ar- chitecture search

    Minbin Huang, Zhijian Huang, Changlin Li, Xin Chen, Hang Xu, Zhenguo Li, and Xiaodan Liang. Arch-graph: Acyclic architecture relation predictor for task-transferable neural ar- chitecture search. InProc. of CVPR, 2022. 3, 8

    2022

  2. [10]

    Flowerformer: Empowering neural architecture encod- ing using a flow-aware graph transformer

    Dongyeong Hwang, Hyunju Kim, Sunwoo Kim, and Kijung Shin. Flowerformer: Empowering neural architecture encod- ing using a flow-aware graph transformer. InProc. of CVPR, pages 6128–6137, 2024. 1, 3, 7

    2024

  3. [11]

    On optimizing top-k metrics for neu- ral ranking models

    Rolf Jagerman, Zhen Qin, Xuanhui Wang, Michael Bender- sky, and Marc Najork. On optimizing top-k metrics for neu- ral ranking models. InProc. of SIGIR, 2022. 8

    2022

  4. [12]

    Graph masked au- toencoder enhanced predictor for neural architecture search

    Kun Jing, Jungang Xu, and Pengfei Li. Graph masked au- toencoder enhanced predictor for neural architecture search. InProc. of IJCAI, 2022. 2, 3, 7, 8

    2022

  5. [13]

    Neural architec- ture search with bayesian optimisation and optimal transport

    Kirthevasan Kandasamy, Willie Neiswanger, Jeff Schnei- der, Barnabas Poczos, and Eric P Xing. Neural architec- ture search with bayesian optimisation and optimal transport. Proc. of NeurIPS, 2018. 7

    2018

  6. [14]

    A new measure of rank correlation

    Maurice G Kendall. A new measure of rank correlation. Biometrika, 30(1/2):81–93, 1938. 3

    1938

  7. [15]

    Semi-supervised classifi- cation with graph convolutional networks

    Thomas N Kipf and Max Welling. Semi-supervised classifi- cation with graph convolutional networks. InProc. of ICLR,

  8. [16]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009. 4

    2009

  9. [17]

    Progressive neural architecture search

    Chenxi Liu, Barret Zoph, Maxim Neumann, Jonathon Shlens, Wei Hua, Li-Jia Li, Li Fei-Fei, Alan Yuille, Jonathan Huang, and Kevin Murphy. Progressive neural architecture search. InProc. of ECCV, 2018. 1, 8

    2018

  10. [18]

    Darts: Differentiable architecture search

    Hanxiao Liu, Karen Simonyan, and Yiming Yang. Darts: Differentiable architecture search. InProc. of ICLR, 2018. 2, 3, 7

    2018

  11. [19]

    Homogeneous architecture augmentation for neural predictor

    Yuqiao Liu, Yehui Tang, and Yanan Sun. Homogeneous architecture augmentation for neural predictor. InProc. of ICCV, 2021. 1, 2

    2021

  12. [20]

    Bridge the gap between architecture spaces via a cross-domain predictor.Proc

    Yuqiao Liu, Yehui Tang, Zeqiong Lv, Yunhe Wang, and Yanan Sun. Bridge the gap between architecture spaces via a cross-domain predictor.Proc. of NeurIPS, 2022. 4, 8

    2022

  13. [21]

    Tnasp: A transformer-based nas predictor with a self- evolution framework.Proc

    Shun Lu, Jixiang Li, Jianchao Tan, Sen Yang, and Ji Liu. Tnasp: A transformer-based nas predictor with a self- evolution framework.Proc. of NeurIPS, 2021. 2, 4, 8

    2021

  14. [22]

    Pinat: A permutation invari- ance augmented transformer for nas predictor

    Shun Lu, Yu Hu, Peihao Wang, Yan Han, Jianchao Tan, Jix- iang Li, Sen Yang, and Ji Liu. Pinat: A permutation invari- ance augmented transformer for nas predictor. InProc. of AAAI, 2023. 1, 2, 4, 6, 8

    2023

  15. [23]

    Neural architecture optimization.Proc

    Renqian Luo, Fei Tian, Tao Qin, Enhong Chen, and Tie-Yan Liu. Neural architecture optimization.Proc. of NeurIPS,

  16. [24]

    A generic graph-based neural architecture encoding scheme for predictor-based nas

    Xuefei Ning, Yin Zheng, Tianchen Zhao, Yu Wang, and Huazhong Yang. A generic graph-based neural architecture encoding scheme for predictor-based nas. InProc. of ECCV,

  17. [25]

    Evaluating ef- ficient performance estimators of neural architectures.Proc

    Xuefei Ning, Changcheng Tang, Wenshuo Li, Zixuan Zhou, Shuang Liang, Huazhong Yang, and Yu Wang. Evaluating ef- ficient performance estimators of neural architectures.Proc. of NeurIPS, 2021. 3

    2021

  18. [26]

    Ta-gates: An encoding scheme for neu- ral network architectures.Proc

    Xuefei Ning, Zixuan Zhou, Junbo Zhao, Tianchen Zhao, Yiping Deng, Changcheng Tang, Shuang Liang, Huazhong Yang, and Yu Wang. Ta-gates: An encoding scheme for neu- ral network architectures.Proc. of NeurIPS, 2022. 3

    2022

  19. [27]

    Efficient neural architecture search via parameters sharing

    Hieu Pham, Melody Guan, Barret Zoph, Quoc Le, and Jeff Dean. Efficient neural architecture search via parameters sharing. InProc. of ICML, 2018. 1

    2018

  20. [28]

    Nas-bench-graph: Benchmarking graph neu- ral architecture search.Proc

    Yijian Qin, Ziwei Zhang, Xin Wang, Zeyang Zhang, and Wenwu Zhu. Nas-bench-graph: Benchmarking graph neu- ral architecture search.Proc. of NeurIPS, 2022. 2, 3, 4

    2022

  21. [29]

    Large-scale evolution of image classifiers

    Esteban Real, Sherry Moore, Andrew Selle, Saurabh Saxena, Yutaka Leon Suematsu, Jie Tan, Quoc V Le, and Alexey Ku- rakin. Large-scale evolution of image classifiers. InProc. of ICML, 2017. 1

    2017

  22. [30]

    Bridging the gap between sample-based and one-shot neural architecture search with bonas.Proc

    Han Shi, Renjie Pi, Hang Xu, Zhenguo Li, James Kwok, and Tong Zhang. Bridging the gap between sample-based and one-shot neural architecture search with bonas.Proc. of NeurIPS, 2020. 1, 3, 4

    2020

  23. [31]

    The lambdaloss framework for ranking metric optimization

    Xuanhui Wang, Cheng Li, Nadav Golbandi, Michael Ben- dersky, and Marc Najork. The lambdaloss framework for ranking metric optimization. InProc. of CIKM, 2018. 8

    2018

  24. [32]

    Textnas: A neural architecture search space tailored for text representation

    Yujing Wang, Yaming Yang, Yiren Chen, Jing Bai, Ce Zhang, Guinan Su, Xiaoyu Kou, Yunhai Tong, Mao Yang, and Lidong Zhou. Textnas: A neural architecture search space tailored for text representation. InProc. of AAAI, 2020. 1

    2020

  25. [33]

    Npenas: Neural predictor guided evo- lution for neural architecture search.IEEE Transactions on Neural Networks and Learning Systems, 34(11):8441–8455,

    Chen Wei, Chuang Niu, Yiping Tang, Yue Wang, Haihong Hu, and Jimin Liang. Npenas: Neural predictor guided evo- lution for neural architecture search.IEEE Transactions on Neural Networks and Learning Systems, 34(11):8441–8455,

  26. [34]

    Neural predictor for neural architecture search

    Wei Wen, Hanxiao Liu, Yiran Chen, Hai Li, Gabriel Ben- der, and Pieter-Jan Kindermans. Neural predictor for neural architecture search. InProc. of ECCV, 2020. 1, 2, 3, 4

    2020

  27. [35]

    Wsabie: Scaling up to large vocabulary image annotation

    Jason Weston, Samy Bengio, and Nicolas Usunier. Wsabie: Scaling up to large vocabulary image annotation. InProc. of IJCAI, 2011. 3

    2011

  28. [36]

    Bananas: Bayesian optimization with neural architectures for neural architecture search

    Colin White, Willie Neiswanger, and Yash Savani. Bananas: Bayesian optimization with neural architectures for neural architecture search. InProc. of AAAI, 2021. 1, 3, 4, 7

    2021

  29. [37]

    How powerful are performance predictors in neural architecture search?Proc

    Colin White, Arber Zela, Robin Ru, Yang Liu, and Frank Hutter. How powerful are performance predictors in neural architecture search?Proc. of NeurIPS, 2021. 5

    2021

  30. [38]

    Stronger nas with weaker predictors

    Junru Wu, Xiyang Dai, Dongdong Chen, Yinpeng Chen, Mengchen Liu, Ye Yu, Zhangyang Wang, Zicheng Liu, Mei Chen, and Lu Yuan. Stronger nas with weaker predictors. Proc. of NeurIPS, 2021. 7, 8

    2021

  31. [39]

    Listwise approach to learning to rank: theory and algorithm

    Fen Xia, Tie-Yan Liu, Jue Wang, Wensheng Zhang, and Hang Li. Listwise approach to learning to rank: theory and algorithm. InProc. of ICML, 2008. 3

    2008

  32. [40]

    Renas: Relativistic evalu- ation of neural architecture search

    Yixing Xu, Yunhe Wang, Kai Han, Yehui Tang, Shangling Jui, Chunjing Xu, and Chang Xu. Renas: Relativistic evalu- ation of neural architecture search. InProc. of CVPR, 2021. 1, 3, 4, 7

    2021

  33. [41]

    Cate: Computation-aware neural architecture encoding with trans- formers

    Shen Yan, Kaiqiang Song, Fei Liu, and Mi Zhang. Cate: Computation-aware neural architecture encoding with trans- formers. InProc. of ICML, 2021. 2, 3, 7

    2021

  34. [42]

    Nar-former: Neural architecture representation learning towards holistic attributes prediction

    Yun Yi, Haokui Zhang, Wenze Hu, Nannan Wang, and Xi- aoyu Wang. Nar-former: Neural architecture representation learning towards holistic attributes prediction. InProc. of CVPR, 2023. 3

    2023

  35. [43]

    Nas-bench-101: Towards reproducible neural architecture search

    Chris Ying, Aaron Klein, Eric Christiansen, Esteban Real, Kevin Murphy, and Frank Hutter. Nas-bench-101: Towards reproducible neural architecture search. InProc. of ICML,

  36. [44]

    Dclp: Neu- ral architecture predictor with curriculum contrastive learn- ing

    Shenghe Zheng, Hongzhi Wang, and Tianyu Mu. Dclp: Neu- ral architecture predictor with curriculum contrastive learn- ing. InProc. of AAAI, 2024. 2, 3, 8

    2024

  37. [45]

    Neural architecture search with reinforcement learning

    Barret Zoph and Quoc Le. Neural architecture search with reinforcement learning. InProc. of ICLR, 2016. 1 10

    2016

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.