Pith. sign in

REVIEW 3 major objections 6 minor 42 references

Language Embedding Meets Dynamic Graph: A New Exploration for Neural Architecture Representation Learning

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read LeDG-Former claims to achieve the first zero-shot latency prediction across hardware platforms by encoding architectures and hardware descriptions in a shared language-model space, and its dynamic graph attention improves topology…

desk verdict A genuinely new combination of LLM text embeddings and dynamic graph attention, but the flagship zero-shot cross-hardware claim is not supported by the experiment as designed. read the letter →

arxiv 2506.07735 v1 pith:ZLKHDMNC submitted 2025-06-09 cs.LG cs.CV

Haizhao Jing , Haokui Zhang , Zhenhao Shang , Rong Xiao , Peng Wang , Yanning Zhang This is my paper
classification cs.LGcs.CV MSC 68T07
keywords neuralarchitecturerepresentationlearninglatencypredictionzero-shotcross-hardwarelanguageembeddingdynamicgraphself-attentionhardware-awaredeploymentsearchtransformer-GNNhybrid
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a neural network's architecture and a hardware platform's specification can both be written as text, embedded by a pretrained language model, and then combined in a transformer so that inference latency can be predicted for hardware never seen during training. If true, engineers could estimate deployment cost on new chips or precision settings without running benchmarks. The proposed framework, LeDG-Former, reports lower average error and higher accuracy within a 10 percent error bound than previous methods on the NNLQP latency benchmark, and it also performs competitively on accuracy prediction for cell-structured NAS-Bench-101 and NAS-Bench-201. The two load-bearing ideas are that BERT-style embeddings create a shared semantic space for architecture and hardware text, and that dynamically selected adjacency masks capture topology better than a fixed adjacency matrix.

What carries the argument

Dynamic Graph Self-Attention (DGSA). This is a transformer attention mechanism where the adjacency structure is not a single fixed matrix. The son mask is the binarized adjacency matrix A, the father mask is Binarize(A^T), and the grandfather mask is Binarize(A^T A^T). For each node, softmax(q_i · k_{1..i}) over the prefix gives a context vector, an MLP maps that vector to three weights W1, W2, W3, and the final node representation is the weighted sum of three attention outputs computed with the three masks plus an identity self-loop term. The other load-bearing piece is the language embedding: a single pretrained language model, BERT in the paper, encodes both operation templates and hardware templates, placing them in the same semantic space so that architecture and platform information can interact in the predictor.

What would settle it

One decisive check: finetune LeDG-Former on T4 under FP32 and INT8 plus P4 under FP32, then zero-shot predict P4 INT8 latency; if accuracy does not beat a linear predictor that receives true hardware features such as measured throughput and precision, the language-embedding semantic space is not doing the transfer work. Another check: replace the pretrained language-model embeddings with randomly initialized embeddings of the same dimension and retrain; if cross-hardware transfer is unchanged, the pretrained semantics are not load-bearing.

Watch

Extended reading notes

Core claim

LeDG-Former is a two-stage framework. In the language embedding stage, each computational node is serialized with a template such as "ParamL Conv 3" and the hardware platform is serialized with a template such as "Nv GPU FP32 8.1 Turing 70W"; both strings pass through the same pretrained language model tokenizer and encoder, producing node embeddings and a platform embedding in a unified space. The platform embedding is concatenated with the architecture representation for final latency prediction. In the representation stage, Dynamic Graph Self-Attention replaces standard self-attention: for each node, the query attends to keys and values under three masks derived from the directed acyclic graph, namely son nodes (direct successors), father nodes (direct predecessors), and grandfather nodes (two-hop predecessors), and a small MLP computes per-node weights for mixing these three attention outputs. The paper argues that this dynamic selection adapts to each node's topological context, and that the language embedding enables cross-hardware transfer, yielding the first successful zero-shot latency prediction across hardware platforms on the NNLQP benchmark.

Load-bearing premise

The load-bearing premise is that a pretrained language model's embeddings of hardware text and architecture text form a shared semantic space in which similarity tracks latency, so that fine-tuning on known hardware transfers to unseen platforms; the paper's zero-shot test only covers a new precision on a seen platform and gives no hardware-feature baseline, so this premise is not directly validated.

Editorial extensions

If this is right

  • Latency can be estimated for a new hardware-precision combination using only text descriptions, without running inference benchmarks on that platform.
  • Hardware-aware deployment and architecture ranking become feasible across heterogeneous devices, since the representation is not tied to one platform.
  • Adding a new platform requires only writing its text template, because the same language model encodes both architecture and hardware information.
  • The dynamic adjacency masks improve latency prediction on NNLQP compared with static adjacency baselines such as NN-Former.
  • The framework transfers to accuracy prediction on cell-structured NAS benchmarks, though the gains there are smaller than on full-network latency prediction.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The reported zero-shot experiment only transfers between FP32 and INT8 on two NVIDIA GPUs; the broader claim of transfer across diverse hardware would be stronger if tested on a genuinely different platform such as a phone NPU or a TPU, which the paper's own conclusion acknowledges is missing.
  • A direct test of the shared-semantic-space assumption would be to train the model with hardware embeddings replaced by randomly initialized vectors of the same dimension; if cross-hardware transfer survives, the pretrained language model is not the mechanism doing the work.
  • The manuscript adopts BERT without citing a reference for it, which leaves the provenance of the load-bearing embeddings unverifiable and makes the ablation against randomly initialized BERT harder to interpret.
  • The dynamic graph mechanism could be applied to other DAG-structured prediction problems, such as program execution time or circuit delay, where two-hop connectivity may also matter.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes LeDG-Former, a neural architecture representation learning framework that combines language-model-based embeddings of both network architectures and hardware specifications with a dynamic graph self-attention mechanism. The method is evaluated on latency prediction using the NNLQP benchmark under in-domain, out-of-domain, and cross-hardware settings, and on accuracy prediction using NAS-Bench-101 and NAS-Bench-201. The authors report state-of-the-art results on NNLQP and competitive results on the NAS-Bench datasets, and claim the first successful zero-shot latency prediction across diverse hardware platforms.

Significance. If the central zero-shot claim is supported, the paper would make a useful contribution by demonstrating that language-based hardware embeddings can enable latency prediction for previously unseen hardware-precision combinations, which is practically relevant for hardware-aware neural architecture search and deployment. The paper also introduces a dynamic graph self-attention mechanism that is a reasonable extension of existing transformer-GNN hybrids. Strengths of the manuscript include the systematic comparison against strong baselines, the use of standard benchmarks, reporting of both average and best results over 10 runs, and ablations that isolate the contributions of the dynamic graph attention and the pretrained language embedding. The main weakness is that the flagship cross-hardware zero-shot claim is not actually supported by the experimental protocol in Section 4.2, and the role of the language-based hardware embedding is not isolated from alternative hardware-feature encodings.

major comments (3)
  1. [Section 4.2, Table 3] The central claim of "zero-shot latency prediction across diverse hardware platforms" is not supported by the reported protocol. As described in the text, the P4→T4 experiment finetunes on P4 FP32, P4 INT8, and T4 INT8, then evaluates on T4 FP32. The target hardware platform T4 is therefore present in the finetuning set; only the precision combination is unseen. This is precision adaptation on a seen platform, not cross-hardware zero-shot prediction. The Table 3 caption further states that "P4→T4 means using latency sample on Tesla P4 for finetune, and zero-shot prediction on Tesla P4 sample," which directly contradicts the text and the column heading. The abstract and conclusion overstate the result by claiming the first successful cross-hardware zero-shot prediction. The authors should either re-run the experiment with a hardware platform entirely absent from training and finetuning, or explicitly reframe the claim as zero-shot precision adaptation on a seen platform.
  2. [Section 4.2] The contribution of the language-based hardware embedding is not isolated. All learning-based baselines (NN-Former) do not use any hardware features, while the proposed method uses language embeddings of hardware text. To support the claim that the unified semantic space is responsible for the observed gains, the authors should add an ablation or baseline that supplies hardware information through a non-language encoder (e.g., a one-hot platform identifier, or handcrafted hardware features such as FLOPs/throughput) under the same dynamic graph architecture. Without such a control, the improvement over NN-Former could be due simply to the extra hardware-conditioning signal rather than to the specific language-based projection.
  3. [Section 4.2 and Section 4.1] The pretraining data may contaminate the zero-shot evaluation. The paper states that the model is pretrained on the NNLQP "unseen" datasets using the same data as in Section 4.1, but it does not state whether the target hardware platform, the target precision, or any samples from the reorganized multi-platform dataset are excluded from pretraining. If the pretraining set contains any latency samples from the target platform or target precision, the reported zero-shot numbers are inflated. The authors must explicitly describe the data split and confirm that all samples from the held-out hardware-precision combination are excluded from every training stage.
minor comments (6)
  1. [Section 4.1, Table 2 description] The text says the training set uses the first 1,800 samples from each of the ten network types and the "remaining 2,000 networks" are used as the test set, but NNLQP has 2,000 samples per type; with 1,800 used for training, only 200 per type should remain. Please correct this numeric inconsistency.
  2. [Section 4.2] The text reports an Acc(10%) of 36.62% for the P4→T4 setting, whereas Table 3 lists an average of 34.62%. One of these is incorrect and should be fixed.
  3. [Section 3.2, Eq. (5)] The summation index i in f_r_nodei = sum_i W_i X_i collides with the node index i; using j as the summation index over the three attention heads would improve clarity.
  4. [Section 3.2, Eqs. (6)-(8)] The symbol σ is used without definition; it should be identified as softmax, and the dimensions of the masks M_Grandfather, M_Father, and M_Son should be stated explicitly.
  5. [Table 4] The column headers in Table 4 are difficult to parse; the relationship between the column numbers and the descriptions (e.g., "Global Attention", "Dynamic Graph Attention", "Position Embedding") should be clarified.
  6. [References] Reference [31] for NN-Former lacks publication venue and year; it should be completed for the paper to be reproducible.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: LeDG-Former is an empirical, supervised pipeline; the Section 4.2 zero-shot protocol is an evaluation-validity concern, not a circularity.

full rationale

This paper does not attempt a first-principles derivation; LeDG-Former is an encoder-plus-predictor trained with supervised losses and evaluated on external benchmarks (NNLQP, NAS-Bench-101, NAS-Bench-201). Equations (1)-(2) define language embeddings and Equations (3)-(8) define dynamic graph self-attention; none is defined in terms of the latency/accuracy labels being predicted. Ablations (Table 4) compare fixed versus dynamic adjacency and randomly initialized versus pretrained BERT embeddings, which is a controlled empirical comparison rather than a fitted parameter renamed as a prediction. The self-citations (NAR-Former [8], NAR-Former V2 [9], NN-Former [31]) are used as design context and baselines, not as an imported uniqueness theorem or an unverified premise, so they are not load-bearing. The substantive concern is in Section 4.2 and Table 3: the text states the P4-to-T4 experiment finetunes on T4 INT8 and then evaluates on T4 FP32, so the target hardware platform is present in finetuning and only the precision combination is unseen; the caption's phrase 'zero-shot prediction on Tesla P4 sample' is internally contradictory, and the conclusion's own limitation statement concedes the cross-hardware dataset is narrow. This undermines the cross-hardware generalization claim as an evaluation-protocol matter, but it is not a circular step in which a prediction or result is equivalent to its inputs by construction.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The central claims rest on domain assumptions about the semantic capacity of BERT embeddings and the sufficiency of hand-designed templates and fixed graph contexts. No ad hoc numerical constants are fitted to make a derivation work.

assumptions (4)
  • domain assumption BERT tokenization and embeddings map heterogeneous textual descriptions into a semantic space where distance correlates with latency-relevant similarity.
    Central to the cross-hardware zero-shot claim; not directly tested (Section 3.1, Section 4.2).
  • domain assumption The hand-designed language templates such as 'ParamL Conv 3' and 'Nv GPU FP32 8.1 Turing 70W' preserve sufficient information for latency prediction.
    Template design is ad hoc; no sensitivity analysis is provided (Section 3.1).
  • domain assumption The three fixed graph contexts (grandfather, father, son) are sufficient, and the dynamic weighting improves over static adjacency.
    No principled justification beyond an ablation on one family (Section 3.2, Table 4).
  • domain assumption Pretraining on NNLQP unseen data followed by finetuning transfers to new hardware-precision combinations.
    Protocol in Section 4.2; no theoretical guarantee.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Language Embedding Meets Dynamic Graph: A New Exploration for Neural Architecture Representation Learning." pith.science (2026). https://pith.science/paper/ZLKHDMNC

@misc{pith2026250607735,
  author       = {Pith},
  title        = {Pith review of: Language Embedding Meets Dynamic Graph: A New Exploration for Neural Architecture Representation Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZLKHDMNC}},
  note         = {Machine review of arXiv:2506.07735}
}
read the original abstract

Neural Architecture Representation Learning aims to transform network models into feature representations for predicting network attributes, playing a crucial role in deploying and designing networks for real-world applications. Recently, inspired by the success of transformers, transformer-based models integrated with Graph Neural Networks (GNNs) have achieved significant progress in representation learning. However, current methods still have some limitations. First, existing methods overlook hardware attribute information, which conflicts with the current trend of diversified deep learning hardware and limits the practical applicability of models. Second, current encoding approaches rely on static adjacency matrices to represent topological structures, failing to capture the structural differences between computational nodes, which ultimately compromises encoding effectiveness. In this paper, we introduce LeDG-Former, an innovative framework that addresses these limitations through the synergistic integration of language-based semantic embedding and dynamic graph representation learning. Specifically, inspired by large language models (LLMs), we propose a language embedding framework where both neural architectures and hardware platform specifications are projected into a unified semantic space through tokenization and LLM processing, enabling zero-shot prediction across different hardware platforms for the first time. Then, we propose a dynamic graph-based transformer for modeling neural architectures, resulting in improved neural architecture modeling performance. On the NNLQP benchmark, LeDG-Former surpasses previous methods, establishing a new SOTA while demonstrating the first successful cross-hardware latency prediction capability. Furthermore, our framework achieves superior performance on the cell-structured NAS-Bench-101 and NAS-Bench-201 datasets.

Figures

Figures reproduced from arXiv: 2506.07735 by the authors.

Figure 1
Figure 1. Overview of the proposed LeDG-Former [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Illustration of the proposed language embedding [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Diagram of the proposed Dynamic Graph Self-Attention (DGSA). [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

42 extracted references · 24 canonical work pages

  1. [1]

    {TVM}: An automated {End-to-End} optimizing compiler for deep learning

    Tianqi Chen, Thierry Moreau, Ziheng Jiang, Lianmin Zheng, Eddie Yan, Haichen Shen, Meghan Cowan, Leyuan Wang, Yuwei Hu, Luis Ceze, et al. {TVM}: An automated {End-to-End} optimizing compiler for deep learning. In13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18), pages 578–594, 2018

    2018

  2. [2]

    Brp-nas: Prediction-based nas using gcns.Advances in neural information processing systems, 33:10480–10490, 2020

    Lukasz Dudziak, Thomas Chau, Mohamed Abdelfattah, Royson Lee, Hyeji Kim, and Nicholas Lane. Brp-nas: Prediction-based nas using gcns.Advances in neural information processing systems, 33:10480–10490, 2020

    work page 2020

  3. [3]

    Nn-meter: Towards accurate latency prediction of deep-learning model inference on diverse edge devices

    Li Lyna Zhang, Shihao Han, Jianyu Wei, Ningxin Zheng, Ting Cao, Yuqing Yang, and Yunxin Liu. Nn-meter: Towards accurate latency prediction of deep-learning model inference on diverse edge devices. InProceedings of the 19th Annual International Conference on Mobile Systems, Applications, and Services, pages 81–93, 2021

    work page 2021

  4. [4]

    Nnlqp: A multi- platform neural network latency query and prediction system with an evolving database

    Liang Liu, Mingzhu Shen, Ruihao Gong, Fengwei Yu, and Hailong Yang. Nnlqp: A multi- platform neural network latency query and prediction system with an evolving database. In Proceedings of the 51st International Conference on Parallel Processing, pages 1–14, 2022

    work page 2022

  5. [5]

    Tfx: A tensorflow-based production-scale machine learning platform

    Denis Baylor, Eric Breck, Heng-Tze Cheng, Noah Fiedel, Chuan Yu Foo, Zakaria Haque, Salem Haykal, Mustafa Ispir, Vihan Jain, Levent Koc, et al. Tfx: A tensorflow-based production-scale machine learning platform. InProceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1387–1395, 2017

    work page 2017

  6. [6]

    Neural predictor for neural architecture search

    Wei Wen, Hanxiao Liu, Yiran Chen, Hai Li, Gabriel Bender, and Pieter-Jan Kindermans. Neural predictor for neural architecture search. InEuropean conference on computer vision, pages 660–676. Springer, 2020

    work page 2020

  7. [7]

    A generic graph-based neural architecture encoding scheme for predictor-based nas

    Xuefei Ning, Yin Zheng, Tianchen Zhao, Yu Wang, and Huazhong Yang. A generic graph-based neural architecture encoding scheme for predictor-based nas. InEuropean Conference on Computer Vision, pages 189–204. Springer, 2020

    work page 2020

  8. [8]

    Nar-former: Neural architecture representation learning towards holistic attributes prediction

    Yun Yi, Haokui Zhang, Wenze Hu, Nannan Wang, and Xiaoyu Wang. Nar-former: Neural architecture representation learning towards holistic attributes prediction. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7715–7724, 2023

    work page 2023

Show all 42 references
  1. [9]

    Nar-former v2: Re- thinking transformer for universal neural network representation learning.Advances in Neural Information Processing Systems, 36:62727–62739, 2023

    Yun Yi, Haokui Zhang, Rong Xiao, Nannan Wang, and Xiaoyu Wang. Nar-former v2: Re- thinking transformer for universal neural network representation learning.Advances in Neural Information Processing Systems, 36:62727–62739, 2023

    2023

  2. [10]

    Neural architecture optimiza- tion.Advances in neural information processing systems, 31, 2018

    Renqian Luo, Fei Tian, Tao Qin, Enhong Chen, and Tie-Yan Liu. Neural architecture optimiza- tion.Advances in neural information processing systems, 31, 2018

    2018

  3. [11]

    Once-for-all: Train one network and specialize it for efficient deployment.arXiv preprint arXiv:1908.09791, 2019

    Han Cai, Chuang Gan, Tianzhe Wang, Zhekai Zhang, and Song Han. Once-for-all: Train one network and specialize it for efficient deployment.arXiv preprint arXiv:1908.09791, 2019

    1908 arXiv

  4. [12]

    Semi-supervised neural architecture search.Advances in Neural Information Processing Systems, 33:10547– 10557, 2020

    Renqian Luo, Xu Tan, Rui Wang, Tao Qin, Enhong Chen, and Tie-Yan Liu. Semi-supervised neural architecture search.Advances in Neural Information Processing Systems, 33:10547– 10557, 2020

    2020

  5. [13]

    Renas: Relativistic evaluation of neural architecture search

    Yixing Xu, Yunhe Wang, Kai Han, Yehui Tang, Shangling Jui, Chunjing Xu, and Chang Xu. Renas: Relativistic evaluation of neural architecture search. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4411–4420, 2021

    2021

  6. [14]

    Contrastive neural architecture search with neural architecture comparators

    Yaofo Chen, Yong Guo, Qi Chen, Minli Li, Wei Zeng, Yaowei Wang, and Mingkui Tan. Contrastive neural architecture search with neural architecture comparators. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9502–9511, 2021

    2021

  7. [15]

    Nas-bench-graph: Benchmarking graph neural architecture search.Advances in neural information processing systems, 35:54–69, 2022

    Yijian Qin, Ziwei Zhang, Xin Wang, Zeyang Zhang, and Wenwu Zhu. Nas-bench-graph: Benchmarking graph neural architecture search.Advances in neural information processing systems, 35:54–69, 2022. 10

    2022

  8. [16]

    Proxylessnas: Direct neural architecture search on target task and hardware.arXiv preprint arXiv:1812.00332, 2018

    Han Cai, Ligeng Zhu, and Song Han. Proxylessnas: Direct neural architecture search on target task and hardware.arXiv preprint arXiv:1812.00332, 2018

    2018 arXiv

  9. [17]

    Understanding and robustifying differentiable architecture search.arXiv preprint arXiv:1909.09656, 2019

    Arber Zela, Thomas Elsken, Tonmoy Saikia, Yassine Marrakchi, Thomas Brox, and Frank Hutter. Understanding and robustifying differentiable architecture search.arXiv preprint arXiv:1909.09656, 2019

    1909 arXiv

  10. [18]

    Neural graph embedding for neural architecture search

    Wei Li, Shaogang Gong, and Xiatian Zhu. Neural graph embedding for neural architecture search. InProceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 4707–4714, 2020

    2020

  11. [19]

    Pace: A parallelizable computation encoder for directed acyclic graphs

    Zehao Dong, Muhan Zhang, Fuhai Li, and Yixin Chen. Pace: A parallelizable computation encoder for directed acyclic graphs. InInternational conference on machine learning, pages 5360–5377. PMLR, 2022

    2022

  12. [20]

    Transformers over directed acyclic graphs.Advances in Neural Information Processing Systems, 36:47764–47782, 2023

    Yuankai Luo, Veronika Thost, and Lei Shi. Transformers over directed acyclic graphs.Advances in Neural Information Processing Systems, 36:47764–47782, 2023

    2023

  13. [21]

    Peephole: Predicting network performance before training.arXiv preprint arXiv:1712.03351, 2017

    Boyang Deng, Junjie Yan, and Dahua Lin. Peephole: Predicting network performance before training.arXiv preprint arXiv:1712.03351, 2017

    2017 arXiv

  14. [22]

    Bananas: Bayesian optimization with neural architectures for neural architecture search

    Colin White, Willie Neiswanger, and Yash Savani. Bananas: Bayesian optimization with neural architectures for neural architecture search. InProceedings of the AAAI conference on artificial intelligence, volume 35, pages 10293–10301, 2021

    2021

  15. [23]

    Semi-supervised classification with graph convolutional networks.arXiv preprint arXiv:1609.02907, 2016

    Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks.arXiv preprint arXiv:1609.02907, 2016

    2016 arXiv

  16. [24]

    Graph attention networks.arXiv preprint arXiv:1710.10903, 2017

    Petar Veliˇckovi´c, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks.arXiv preprint arXiv:1710.10903, 2017

    2017 arXiv

  17. [25]

    Tnasp: A transformer-based nas predictor with a self-evolution framework.Advances in Neural Information Processing Systems, 34:15125–15137, 2021

    Shun Lu, Jixiang Li, Jianchao Tan, Sen Yang, and Ji Liu. Tnasp: A transformer-based nas predictor with a self-evolution framework.Advances in Neural Information Processing Systems, 34:15125–15137, 2021

    2021

  18. [26]

    Nas-bench-101: Towards reproducible neural architecture search

    Chris Ying, Aaron Klein, Eric Christiansen, Esteban Real, Kevin Murphy, and Frank Hutter. Nas-bench-101: Towards reproducible neural architecture search. InInternational conference on machine learning, pages 7105–7114. PMLR, 2019

    2019

  19. [27]

    Nas-bench-201: Extending the scope of reproducible neural architecture search.arXiv preprint arXiv:2001.00326, 2020

    Xuanyi Dong and Yi Yang. Nas-bench-201: Extending the scope of reproducible neural architecture search.arXiv preprint arXiv:2001.00326, 2020

    2001 arXiv

  20. [28]

    Do transformers really perform badly for graph representation?Advances in neural information processing systems, 34:28877–28888, 2021

    Chengxuan Ying, Tianle Cai, Shengjie Luo, Shuxin Zheng, Guolin Ke, Di He, Yanming Shen, and Tie-Yan Liu. Do transformers really perform badly for graph representation?Advances in neural information processing systems, 34:28877–28888, 2021

    2021

  21. [29]

    Representing long-range context for graph neural networks with global attention

    Zhanghao Wu, Paras Jain, Matthew Wright, Azalia Mirhoseini, Joseph E Gonzalez, and Ion Stoica. Representing long-range context for graph neural networks with global attention. Advances in neural information processing systems, 34:13266–13279, 2021

    2021

  22. [30]

    A neural architecture predictor based on gnn- enhanced transformer

    Xunzhi Xiang, Kun Jing, and Jungang Xu. A neural architecture predictor based on gnn- enhanced transformer. InInternational Conference on Artificial Intelligence and Statistics, pages 1729–1737. PMLR, 2024

    2024

  23. [31]

    Nn-former: Rethinking graph structure in neural architecture representation

    Ruihan Xu, Haokui Zhang, and Shiliang Zhang. Nn-former: Rethinking graph structure in neural architecture representation

  24. [32]

    Does unsupervised architecture representation learning help neural architecture search?Advances in neural information processing systems, 33:12486–12498, 2020

    Shen Yan, Yu Zheng, Wei Ao, Xiao Zeng, and Mi Zhang. Does unsupervised architecture representation learning help neural architecture search?Advances in neural information processing systems, 33:12486–12498, 2020

    2020

  25. [33]

    Ta-gates: an encoding scheme for neural network architectures

    Xuefei Ning, Zixuan Zhou, Junbo Zhao, Tianchen Zhao, Yiping Deng, Changcheng Tang, Shuang Liang, Huazhong Yang, and Yu Wang. Ta-gates: an encoding scheme for neural network architectures. InProceedings of the 36th International Conference on Neural Information Processing Syste...

    2022

  26. [34]

    The graph neural network model.IEEE transactions on neural networks, 20(1):61–80, 2008

    Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. The graph neural network model.IEEE transactions on neural networks, 20(1):61–80, 2008

    2008

  27. [35]

    Representation learning on graphs: Methods and applications.arXiv preprint arXiv:1709.05584, 2017

    William L Hamilton, Rex Ying, and Jure Leskovec. Representation learning on graphs: Methods and applications.arXiv preprint arXiv:1709.05584, 2017

    2017 arXiv

  28. [36]

    How powerful are graph neural networks?arXiv preprint arXiv:1810.00826, 2018

    Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks?arXiv preprint arXiv:1810.00826, 2018

    2018 arXiv

  29. [37]

    Benchmarking graph neural networks.Journal of Machine Learning Research, 24(43):1–48, 2023

    Vijay Prakash Dwivedi, Chaitanya K Joshi, Anh Tuan Luu, Thomas Laurent, Yoshua Bengio, and Xavier Bresson. Benchmarking graph neural networks.Journal of Machine Learning Research, 24(43):1–48, 2023

    2023

  30. [38]

    Neural architecture search with reinforcement learning.arXiv preprint arXiv:1611.01578, 2016

    Barret Zoph and Quoc V Le. Neural architecture search with reinforcement learning.arXiv preprint arXiv:1611.01578, 2016

    2016 arXiv

  31. [39]

    Darts: Differentiable architecture search

    Hanxiao Liu, Karen Simonyan, and Yiming Yang. Darts: Differentiable architecture search. arXiv preprint arXiv:1806.09055, 2018

    2018 arXiv

  32. [40]

    Autogt: Automated graph transformer architecture search

    Zizhao Zhang, Xin Wang, Chaoyu Guan, Ziwei Zhang, Haoyang Li, and Wenwu Zhu. Autogt: Automated graph transformer architecture search. InThe Eleventh International Conference on Learning Representations, 2023

    2023

  33. [41]

    A learned performance model for tensor processing units.Proceedings of Machine Learning and Systems, 3:387–400, 2021

    Sam Kaufman, Phitchaya Phothilimthana, Yanqi Zhou, Charith Mendis, Sudip Roy, Amit Sabne, and Mike Burrows. A learned performance model for tensor processing units.Proceedings of Machine Learning and Systems, 3:387–400, 2021

    2021

  34. [42]

    Pinat: a permutation invariance augmented transformer for nas predictor

    Shun Lu, Yu Hu, Peihao Wang, Yan Han, Jianchao Tan, Jixiang Li, Sen Yang, and Ji Liu. Pinat: a permutation invariance augmented transformer for nas predictor. InProceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 8957–8965, 2023. 12

    2023

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.