Pith. sign in

REVIEW 3 major objections 3 minor 74 references

NADER: Neural Architecture Design via Multi-Agent Collaboration

T0 review · 3 major / 3 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read NADER shows that multi-agent LLM collaboration can design neural networks that outperform the best architecture inside the predetermined NAS-Bench-201 search space.

desk verdict The framework is novel and clearly described, but the headline result is a test-selection artifact because Algorithm 2 uses test-set accuracy to drive the search. read the letter →

arxiv 2412.19206 v1 pith:DWTTNF76 submitted 2024-12-26 cs.CV

Zekang Yang , Wang Zeng , Sheng Jin , Chen Qian , Ping Luo , Wentao Liu This is my paper · ORCID
classification cs.CV
keywords neuralarchitecturedesignmulti-agentLLMcollaborationagentsNAS-Bench-201graph-basedrepresentationexperiencereflectionopen-endedsearchtest-setguidedselection
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

NADER is a claim about open-ended architecture design: a team of large-language-model agents can improve a base network by proposing structural modifications one step at a time, without being confined to a search space that an expert predefines. The paper reports that this loop finds networks exceeding the best accuracy achievable inside NAS-Bench-201, a standard benchmark search space, on nearly every dataset it tries. For example, ten designed architectures reach 49.63% test accuracy on ImageNet16-120, above the space's 47.31% optimum, and 500 designs reach 76.00% on CIFAR-100, 5.14 points above its ResNet starting point. If correct, this would mean the bottleneck in architecture discovery is not the search algorithm alone but the ceiling imposed by the search space itself.

What carries the argument

The mechanism is a four-agent loop plus a graph-based network representation. Architectures are written as single directed acyclic graphs, with nodes as operations and edges as information flow, so the LLM works on structure rather than code syntax. The Reader distills design ideas from recent papers, the Proposer picks a candidate and a modification suggestion, the Modifier rewrites the graph, and the Reflector checks executability, explains failures, and stores reusable design experience. The modification tree records every attempted architecture and its measured accuracy, and the Proposer navigates that tree to choose which network to develop next.

What would settle it

Retrain the best architectures reported for ImageNet16-120 and CIFAR-100 from scratch while selecting modifications using validation accuracy only, never test accuracy, then evaluate once on the test split; if the resulting accuracy falls to or below the NAS-Bench-201 optimum, test-set selection drove the claimed improvement.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that an LLM-based multi-agent system can treat architecture design as iterative, open-ended modification and, in nearly all benchmark scenarios, produce networks that beat the optimal accuracy reachable inside the NAS-Bench-201 search space. Starting from either a ResNet or a randomly sampled network, NADER runs a loop in which the Proposer selects a promising node in a modification tree, the Modifier edits the network's graph representation, and the Reflector verifies executability and accumulates design experience. With ten architectures designed, test accuracy reaches 49.63% on ImageNet16-120, above the 47.31% optimum; with 500 designs on CIFAR-100 it reaches 76.00%, 5.14 points above its ResNet initialization. The paper also reports that this large run cost only about $0.046 per architecture in LLM tokens.

Load-bearing premise

The search loop reads the test accuracy of each candidate architecture and uses that accuracy to choose the next modification; if using the test set to guide design is not a valid protocol, the reported numbers overstate how well the method generalizes.

Editorial extensions

If this is right

  • If the framework works as claimed, a handful of design cycles (5 to 10) can match or exceed what thousands of evaluations find inside a fixed search space.
  • The graph representation and experience-retrieval design suggest the same agents can be pointed at other building blocks, such as GoogLeNet, ConvNeXt, or SE-ResNet blocks, without redefining a search space.
  • The 500-iteration result implies the method keeps improving with budget, and the resulting architecture transfers to other datasets, including GTSRB and STL-10.
  • The reported success rates and token counts imply the cost of open-ended design is small enough to be a practical alternative to conventional NAS.
  • The paper's graph representation also supports isomorphism checking, so repeated or duplicate architectures are not trained twice.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A validation-only replay of the search would show how much of the reported lead over the NAS-Bench-201 optimum comes from test-set selection rather than from architectural quality.
  • The same loop could be run with accuracy replaced by latency or memory, turning the design objective into any measurable property; the architecture graph is a convenient substrate for such multi-objective design.
  • The paper's own observation that 19.2% of 500 designs fail to converge suggests a cheap early-stopping signal based on loss behavior in the first epochs could make the search considerably cheaper, an optimization the authors mention but do not formalize.
  • Because the Reader ingests recent papers, the framework's knowledge base will age; tracking which retrieved inspirations actually cause accuracy gains would let the system prune stale knowledge automatically.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. The paper proposes NADER, a multi-agent LLM framework for neural architecture design (NAD) that iteratively modifies a base architecture using a Reader, Proposer, Modifier, and Reflector, with a graph-based architecture representation and an experience database for reflection. The authors claim that NADER discovers architectures that, in nearly all scenarios, surpass the optimal accuracy achievable within the NAS-Bench-201 search space, using only 5-10 generated architectures, and they report large-scale experiments on CIFAR-100 reaching 76.0% test accuracy. The paper also introduces a benchmark for evaluating LLM-based architecture modification and reports ablation studies on the framework components.

Significance. If the reported results were valid, the framework would be a notable contribution: it combines multi-agent LLM collaboration, graph-based architecture representation, and reflection from both immediate feedback and stored experience to explore architectures outside predefined search spaces, and it explicitly credits the Reader/Proposer/Reflector design and the graph representation as enabling more efficient design than code-level generation. The NAD benchmark and the systematic ablation studies (Tables 2 and 3) are useful steps. However, the empirical core of the paper is undermined by the use of test-set accuracy to guide the search, which makes the claimed superiority over NAS-Bench-201 baselines an artifact of test-set selection rather than an unbiased estimate of generalization. The significance of the framework itself can only be assessed after the evaluation protocol is corrected.

major comments (3)
  1. [Algorithm 2 / Section 3.3.2 / Appendix A3.2] The search objective is the test-set accuracy. Algorithm 2 lines 15-17 reads 'arch_acc_n <- Train architecture arch_n on train dataset and get its test accuracy on test dataset' and then stores that accuracy in the modification tree, while Section 3.3.2 states that the Proposer 'prioritizes nodes with higher performance scores' when navigating the tree. The test set is therefore queried repeatedly during search, and the final architecture is selected on the basis of these queries. Appendix A3.2 confirms this by describing the distribution of test accuracy of 500 models and selecting the optimal model on the CIFAR-100 test set. Consequently, the reported test accuracies are not independent estimates of generalization, and the core claim that NADER 'surpasses the optimal accuracy achievable within the predetermined search space' (Section 4.2) is not supported. A corrected protocol should perform architecture selection on the validation split and evaluate the final architecture on the test set exactly once.
  2. [Table 1] The comparisons in Table 1 are unfair because NADER selects on test accuracy while the NAS baselines (RS, REA, BOHB, GENIUS, LLMatic, etc.) and the 'Optimal' row are computed with validation-based selection or are the known test accuracy of the best architecture in the search space. The reported numbers show the expected signature of test-set selection: validation accuracy is sometimes lower than test accuracy for the same method (e.g., Random initialization, 10 architectures on CIFAR-10: 91.16 validation vs 94.40 test; ResNet initialization, 500 architectures on CIFAR-100: 75.72 validation vs 76.00 test). Selecting the maximum of a set of noisy test evaluations can push the best test accuracy above both the validation accuracy and the NAS-Bench-201 'Optimal' test accuracy, so the claimed superiority over the search-space optimum is not evidence of out-of-search-space performance.
  3. [Section 4.4 / Figure 4 / Appendix A3.3] The large-scale NAD experiment is also affected by the same protocol problem. Section 4.4 reports that the final architecture achieves 76.0% on the test set, and Figure 4 colors nodes by test accuracy; Appendix A3.2 explicitly states that the distribution of test accuracy of 500 models is used and that the optimal model is selected on the CIFAR-100 test set. The consequent cross-dataset generalization evaluation in A3.3 (NADER-500 evaluated on GTSRB and STL-10) is not a clean test of generalization, because the architecture itself was selected using the CIFAR-100 test set. A validation-based rerun of the large-scale experiment is needed before any conclusions about the method's ability to discover architectures beyond the search space can be drawn.
minor comments (3)
  1. [Table 2] The column headers in Table 2 run together ('validation test validation test'), making the table hard to read; please reformat with clear separators.
  2. [Various places] There are several typos: 'paramerts' for 'parameters' in Section 4.2, 'Futhermore, we evalute' in Section 4.4, 'Trail' for 'Trial' in Appendix A3.1, and 'Algorithm 2 shows the design process of neural architecture' in Section 3.1 appears to be missing the word 'design'. Please proofread.
  3. [Table 1 caption] The caption says 'We report the mean validation and test accuracies for 5 three runs'; the phrase '5 three runs' is unclear and should be rewritten, e.g., 'for three runs per setting'.

Circularity Check

1 steps flagged · score 8.0 of 10

NADER's central claim that designed architectures surpass the NAS-Bench-201 optimum is not an independent result: the search selects candidates using their test-set accuracies, so the reported test accuracy of the chosen architecture is a test-selection artifact.

  1. fitted input called prediction [Algorithm 2, lines 16-17 and 22; Section 3.3.2; Section 4.2; Appendix A3.2]
    "arch accn ← Train architecture archn on train dataset and get its test accuracy on test dataset. ... T ←Update T with prop, archn and arch accn. ... archbest ← Get the best architecture from {archi}N i=1. ... The Proposer leverages the performance data recorded within the graph to guide its selection. It prioritizes nodes with higher performance scores ... in nearly all scenarios, NADER consistently designs architectures that surpass the optimal accuracy achievable within the predetermined search space."

    The reported test accuracy is both the search signal and the reported outcome. Algorithm 2 records each candidate's test accuracy in the modification tree, and Section 3.3.2 says the Proposer prioritizes nodes with higher recorded performance; the final architecture is then selected as the best among the test-evaluated candidates. With only 10 test-set queries on ImageNet16-120, NADER reports 49.63% test accuracy versus the 47.31% optimum; with 500 queries on CIFAR-100 it reports 76.00% test accuracy against 75.72% validation. Selecting the maximum of a set of test-evaluated models inflates the expectation of the reported test accuracy, so the comparison against the NAS-Bench-201 'Optimal' row and against validation-selecting baselines is biased by construction.

full rationale

The paper's framework itself is not definitionally circular: the multi-agent loop, graph representation, and Reflector are substantive engineering contributions, and the NAD benchmark metrics (executability, quality, success rate) are evaluated independently of the search objective. However, the paper's strongest empirical claim — that NADER designs architectures that exceed the optimal accuracy achievable within the NAS-Bench-201 search space — is undermined by a direct test-set feedback loop. Algorithm 2 line 16 obtains the test accuracy of every generated candidate; line 17 stores that number in the modification tree; Section 3.3.2 instructs the Proposer to prioritize nodes with higher recorded performance; and line 22 returns the best architecture from the test-evaluated set. The reported test accuracies in Table 1 and Section 4.4 are therefore maxima over test-evaluated candidates, not holdout evaluations of a search that used only training/validation signals. This is the pattern of a fitted input being reported as a prediction: the test set is effectively used for model selection, so the reported gain over the NAS-Bench-201 'Optimal' value (which comes from a search space evaluated without such test-set feedback) is statistically forced upward. The large-scale run makes the artifact visible: validation accuracy is 75.72% while test accuracy is 76.00%, and Appendix A3.2 explicitly plots the distribution of test accuracies of the 500 designed models. No self-citation or imported uniqueness theorem is load-bearing here; the circularity is in the evaluation protocol. The correct remedy is a validation-only selection protocol, after which the final test accuracy would be an honest estimate. Because the central headline result is compromised by this construction, the circularity score is high, though the method's non-empirical components retain independent content.

Assumptions & free parameters 2 free parameters · 2 assumptions · 0 invented entities

The central claim depends mainly on the performance of the LLM and on the correctness of the evaluation protocol. The LLM's pretrained knowledge is a model-internal assumption with no external evidence; the test-set usage is a methodological assumption that is both unstated in the fairness section and fatal to the interpretation of the results. The method introduces no physical entities or new measured constants.

free parameters (2)
  • number of retrieved experiences = 5
    The Reflector retrieves 5 relevant design experiences per query (Section 3.4.2). This value is chosen by hand and not justified by a sensitivity study.
  • search budget N = 5, 10, 500
    The number of generated architectures is an experimental setting, not fitted, but it directly affects the reported best accuracy. A larger N always improves results, so comparisons at different N are not apples-to-apples.
assumptions (2)
  • domain assumption GPT-4o can reliably generate and modify DAG-based architecture descriptions
    The whole framework rests on the LLM's ability to follow the graph format and produce semantically valid modifications. No independent verification of this competency is provided beyond the self-built benchmark.
  • domain assumption Test-set accuracy is an acceptable search signal
    Algorithm 2 line 16 explicitly records test accuracy into the modification tree and the Proposer uses this performance to select future candidates. This assumes that test-set peeking is a valid protocol, which contradicts the standard NAS-Bench-201 practice of selecting on validation and reporting test only at the end.

how reviews work

0 comments
Cite this review

Pith. "Pith review of NADER: Neural Architecture Design via Multi-Agent Collaboration." pith.science (2026). https://pith.science/paper/DWTTNF76

@misc{pith2026241219206,
  author       = {Pith},
  title        = {Pith review of: NADER: Neural Architecture Design via Multi-Agent Collaboration},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DWTTNF76}},
  note         = {Machine review of arXiv:2412.19206}
}
read the original abstract

Designing effective neural architectures poses a significant challenge in deep learning. While Neural Architecture Search (NAS) automates the search for optimal architectures, existing methods are often constrained by predetermined search spaces and may miss critical neural architectures. In this paper, we introduce NADER (Neural Architecture Design via multi-agEnt collaboRation), a novel framework that formulates neural architecture design (NAD) as a LLM-based multi-agent collaboration problem. NADER employs a team of specialized agents to enhance a base architecture through iterative modification. Current LLM-based NAD methods typically operate independently, lacking the ability to learn from past experiences, which results in repeated mistakes and inefficient exploration. To address this issue, we propose the Reflector, which effectively learns from immediate feedback and long-term experiences. Additionally, unlike previous LLM-based methods that use code to represent neural architectures, we utilize a graph-based representation. This approach allows agents to focus on design aspects without being distracted by coding. We demonstrate the effectiveness of NADER in discovering high-performing architectures beyond predetermined search spaces through extensive experiments on benchmark tasks, showcasing its advantages over state-of-the-art methods. The codes will be released soon.

Figures

Figures reproduced from arXiv: 2412.19206 by the authors.

Figure 1
Figure 1. Traditional Neural Architecture Search (NAS) ap [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of NADER framework. The Reader continuously learns from academic literature, while the Proposer identifies the [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Illustration of graph-based neural architecture represen [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Large-scale NAD experiments. Each node represents a [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

74 extracted references · 50 canonical work pages

  1. [1]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ah- mad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 ,

    arXiv

  2. [2]

    Random search for hyper-parameter optimization

    James Bergstra and Yoshua Bengio. Random search for hyper-parameter optimization. Journal of machine learning research, 13(2), 2012. 7

    work page 2012

  3. [3]

    Image quality- aware diagnosis via meta-knowledge co-embedding

    Haoxuan Che, Siyu Chen, and Hao Chen. Image quality- aware diagnosis via meta-knowledge co-embedding. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 19819–19829, 2023. 12

    work page 2023

  4. [4]

    Evoprompting: language models for code-level neural architecture search

    Angelica Chen, David Dohan, and David So. Evoprompting: language models for code-level neural architecture search. Advances in Neural Information Processing Systems , 36,

    work page

  5. [5]

    Agentverse: Facilitating multi-agent col- laboration and exploring emergent behaviors in agents.arXiv preprint arXiv:2308.10848, 2023

    Weize Chen, Yusheng Su, Jingwei Zuo, Cheng Yang, Chen- fei Yuan, Chen Qian, Chi-Min Chan, Yujia Qin, Yaxi Lu, Ruobing Xie, et al. Agentverse: Facilitating multi-agent col- laboration and exploring emergent behaviors in agents.arXiv preprint arXiv:2308.10848, 2023. 2

    arXiv 2023

  6. [6]

    DrNAS: Dirichlet Neural Architecture Search

    Xiangning Chen, Ruochen Wang, Minhao Cheng, Xi- aocheng Tang, and Cho-Jui Hsieh. Drnas: Dirichlet neural architecture search. arXiv preprint arXiv:2006.10355, 2020. 2, 7

    work page Pith review arXiv 2006

  7. [7]

    A downsampled variant of imagenet as an alternative to the ci- far datasets

    Patryk Chrabaszcz, Ilya Loshchilov, and Frank Hutter. A downsampled variant of imagenet as an alternative to the ci- far datasets. arXiv preprint arXiv:1707.08819, 2017. 6, 11

    arXiv 2017

  8. [8]

    An analysis of single-layer networks in unsupervised feature learning

    Adam Coates, Honglak Lee, Andrew Y Ng, Adam Coates, Honglak Lee, and Andrew Y Ng. An analysis of single-layer networks in unsupervised feature learning. In Aistats, 2011. 17

    work page 2011

Show all 74 references
  1. [9]

    One-shot neural architecture search via self-evaluated template network

    Xuanyi Dong and Yi Yang. One-shot neural architecture search via self-evaluated template network. In Proceedings of the IEEE/CVF International Conference on Computer Vi- sion, pages 3681–3690, 2019. 2, 7

    2019

  2. [10]

    Searching for a robust neu- ral architecture in four gpu hours

    Xuanyi Dong and Yi Yang. Searching for a robust neu- ral architecture in four gpu hours. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1761–1770, 2019. 2, 7

    2019

  3. [11]

    Nas-bench-201: Extending the scope of reproducible neural architecture search

    Xuanyi Dong and Yi Yang. Nas-bench-201: Extending the scope of reproducible neural architecture search. arXiv preprint arXiv:2001.00326, 2020. 6, 11

    2001 arXiv

  4. [12]

    Bohb: Ro- bust and efficient hyperparameter optimization at scale

    Stefan Falkner, Aaron Klein, and Frank Hutter. Bohb: Ro- bust and efficient hyperparameter optimization at scale. In International conference on machine learning, pages 1437–

  5. [13]

    Visual program- ming: Compositional visual reasoning without training

    Tanmay Gupta and Aniruddha Kembhavi. Visual program- ming: Compositional visual reasoning without training. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 14953–14962, 2023. 2

    2023

  6. [14]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 2, 6, 8

    2016

  7. [15]

    Metagpt: Meta pro- gramming for multi-agent collaborative framework

    Sirui Hong, Xiawu Zheng, Jonathan Chen, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, et al. Metagpt: Meta pro- gramming for multi-agent collaborative framework. arXiv preprint arXiv:2308.00352, 2023. 1, 2

    2023 arXiv

  8. [16]

    Dsnas: Direct neural architecture search without parameter retraining

    Shoukang Hu, Sirui Xie, Hehui Zheng, Chunxiao Liu, Jian- ping Shi, Xunying Liu, and Dahua Lin. Dsnas: Direct neural architecture search without parameter retraining. InProceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12084–12092, 2020. 2, 7

    2020

  9. [17]

    Neural architecture search with bayesian optimisation and optimal transport.Ad- vances in neural information processing systems , 31, 2018

    Kirthevasan Kandasamy, Willie Neiswanger, Jeff Schneider, Barnabas Poczos, and Eric P Xing. Neural architecture search with bayesian optimisation and optimal transport.Ad- vances in neural information processing systems , 31, 2018. 2

    2018

  10. [18]

    Learning multiple layers of features from tiny images

    Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. Master’s thesis, Univer- sity of Tront, 2009. 6, 11, 17

    2009

  11. [19]

    Retrieval-augmented generation for knowledge-intensive nlp tasks

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K¨uttler, Mike Lewis, Wen-tau Yih, Tim Rockt ¨aschel, et al. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing Syst...

    2020

  12. [20]

    Random search and repro- ducibility for neural architecture search

    Liam Li and Ameet Talwalkar. Random search and repro- ducibility for neural architecture search. In Uncertainty in artificial intelligence, pages 367–377. PMLR, 2020. 7

    2020

  13. [21]

    Encouraging divergent thinking in large lan- guage models through multi-agent debate

    Tian Liang, Zhiwei He, Wenxiang Jiao, Xing Wang, Yan Wang, Rui Wang, Yujiu Yang, Zhaopeng Tu, and Shum- ing Shi. Encouraging divergent thinking in large lan- guage models through multi-agent debate. arXiv preprint arXiv:2305.19118, 2023. 2

    2023 arXiv

  14. [22]

    Darts: Differentiable architecture search

    Hanxiao Liu, Karen Simonyan, and Yiming Yang. Darts: Differentiable architecture search. arXiv preprint arXiv:1806.09055, 2018. 1, 2, 7

    2018 arXiv

  15. [23]

    Bolaa: Bench- marking and orchestrating llm-augmented autonomous agents

    Zhiwei Liu, Weiran Yao, Jianguo Zhang, Le Xue, Shelby Heinecke, Rithesh Murthy, Yihao Feng, Zeyuan Chen, Juan Carlos Niebles, Devansh Arpit, et al. Bolaa: Bench- marking and orchestrating llm-augmented autonomous agents. arXiv preprint arXiv:2308.05960, 2023. 2

    2023 arXiv

  16. [24]

    The ai scientist: Towards fully au- 9 tomated open-ended scientific discovery

    Chris Lu, Cong Lu, Robert Lange, Jakob N Foerster, Jeff Clune, and David Ha. The ai scientist: Towards fully au- 9 tomated open-ended scientific discovery. arXiv preprint arXiv:2408.06292, 2024. 1

    2024 arXiv

  17. [25]

    Lambda-darts: Mitigating performance collapse by harmonizing operation selection among cells

    Sajad Movahedi, Melika Adabinejad, Ayyoob Imani, Are- zou Keshavarz, Mostafa Dehghani, Azadeh Shakery, and Babak N Araabi. Lambda-darts: Mitigating performance collapse by harmonizing operation selection among cells. arXiv preprint arXiv:2210.07998, 2022. 7

    arXiv

  18. [26]

    Llmatic: Neural archi- tecture search via large language models and quality diver- sity optimization

    Muhammad U Nasir, Sam Earle, Christopher Cleghorn, Steven James, and Julian Togelius. Llmatic: Neural archi- tecture search via large language models and quality diver- sity optimization. arXiv preprint arXiv:2306.01102 , 2023. 3, 7

    2023 arXiv

  19. [27]

    Generative agents: Interactive simulacra of human behavior

    Joon Sung Park, Joseph O’Brien, Carrie Jun Cai, Mered- ith Ringel Morris, Percy Liang, and Michael S Bernstein. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th annual acm symposium on user interface software and technology, pages 1–22, 2023. 1, 2

    2023

  20. [28]

    Efficient neural architecture search via parameters sharing

    Hieu Pham, Melody Guan, Barret Zoph, Quoc Le, and Jeff Dean. Efficient neural architecture search via parameters sharing. In International conference on machine learning , pages 4095–4104. PMLR, 2018. 2, 7

    2018

  21. [29]

    Commu- nicative agents for software development

    Chen Qian, Xin Cong, Cheng Yang, Weize Chen, Yusheng Su, Juyuan Xu, Zhiyuan Liu, and Maosong Sun. Commu- nicative agents for software development. arXiv preprint arXiv:2307.07924, 6, 2023. 2

    2023 arXiv

  22. [30]

    Toolllm: Facilitating large language models to master 16000+ real-world apis

    Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, et al. Toolllm: Facilitating large language models to master 16000+ real-world apis. arXiv preprint arXiv:2307.16789 ,

    arXiv

  23. [31]

    Lemo- nade: Multi-parameter neural architecture discovery with llms

    Md Hafizur Rahman and Prabuddha Chakraborty. Lemo- nade: Multi-parameter neural architecture discovery with llms. arXiv preprint arXiv:2402.18443, 2024. 7, 8

    2024 arXiv

  24. [32]

    Regularized evolution for image classifier architecture search

    Esteban Real, Alok Aggarwal, Yanping Huang, and Quoc V Le. Regularized evolution for image classifier architecture search. In Proceedings of the aaai conference on artificial intelligence, pages 4780–4789, 2019. 2, 7

    2019

  25. [33]

    Toolformer: Lan- guage models can teach themselves to use tools

    Timo Schick, Jane Dwivedi-Yu, Roberto Dess `ı, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Lan- guage models can teach themselves to use tools. Advances in Neural Information Processing Systems, 36, 2024. 2

    2024

  26. [34]

    Hugginggpt: Solving ai tasks with chatgpt and its friends in hugging face

    Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. Hugginggpt: Solving ai tasks with chatgpt and its friends in hugging face. Advances in Neural Information Processing Systems, 36, 2024. 2

    2024

  27. [35]

    Reflexion: Language agents with verbal reinforcement learning

    Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning. Advances in Neural In- formation Processing Systems, 36, 2024. 2

    2024

  28. [36]

    Stallkamp, M

    J. Stallkamp, M. Schlipsing, J. Salmen, and C. Igel. Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition. Neural Netw, 32:323–332, 2012. 17

    2012

  29. [37]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth´ee Lacroix, Baptiste Rozi`ere, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023. 1

    2023 arXiv

  30. [38]

    Chain-of-thought prompting elicits reasoning in large lan- guage models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large lan- guage models. Advances in neural information processing systems, 35:24824–24837, 2022. 1

    2022

  31. [39]

    Cap- vstnet: Content affinity preserved versatile style transfer

    Linfeng Wen, Chengying Gao, and Changqing Zou. Cap- vstnet: Content affinity preserved versatile style transfer. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition (CVPR), pages 18300–18309,

  32. [40]

    Simple statistical gradient-following al- gorithms for connectionist reinforcement learning

    Ronald J Williams. Simple statistical gradient-following al- gorithms for connectionist reinforcement learning. Machine learning, 8:229–256, 1992. 2, 7

    1992

  33. [41]

    Visual chatgpt: Talking, drawing and editing with visual foundation models

    Chenfei Wu, Shengming Yin, Weizhen Qi, Xiaodong Wang, Zecheng Tang, and Nan Duan. Visual chatgpt: Talking, drawing and editing with visual foundation models. arXiv preprint arXiv:2303.04671, 2023. 2

    2023 arXiv

  34. [42]

    The rise and potential of large language model based agents: A survey

    Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, et al. The rise and potential of large language model based agents: A survey. arXiv preprint arXiv:2309.07864 ,

    arXiv

  35. [43]

    Snas: stochastic neural architecture search

    Sirui Xie, Hehui Zheng, Chunxiao Liu, and Liang Lin. Snas: stochastic neural architecture search. arXiv preprint arXiv:1812.09926, 2018. 2, 7

    2018 arXiv

  36. [44]

    Pc-darts: Partial channel connections for memory-efficient architecture search

    Yuhui Xu, Lingxi Xie, Xiaopeng Zhang, Xin Chen, Guo-Jun Qi, Qi Tian, and Hongkai Xiong. Pc-darts: Partial channel connections for memory-efficient architecture search. arXiv preprint arXiv:1907.05737, 2019. 2, 7

    1907 arXiv

  37. [45]

    Autommlab: Automatically generating de- ployable models from language instructions for computer vi- sion tasks

    Zekang Yang, Wang Zeng, Sheng Jin, Chen Qian, Ping Luo, and Wentao Liu. Autommlab: Automatically generating de- ployable models from language instructions for computer vi- sion tasks. arXiv preprint arXiv:2402.15351, 2024. 1

    2024 arXiv

  38. [46]

    b-darts: Beta-decay regularization for differentiable architecture search

    Peng Ye, Baopu Li, Yikang Li, Tao Chen, Jiayuan Fan, and Wanli Ouyang. b-darts: Beta-decay regularization for differentiable architecture search. In proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10874–10883, 2022. 2, 7

    2022

  39. [47]

    idarts: Differen- tiable architecture search with stochastic implicit gradients

    Miao Zhang, Steven W Su, Shirui Pan, Xiaojun Chang, Ehsan M Abbasnejad, and Reza Haffari. idarts: Differen- tiable architecture search with stochastic implicit gradients. In International Conference on Machine Learning , pages 12557–12566. PMLR, 2021. 2, 7

    2021

  40. [48]

    Using large language models for hy- perparameter optimization

    Michael R Zhang, Nishkrit Desai, Juhan Bae, Jonathan Lor- raine, and Jimmy Ba. Using large language models for hy- perparameter optimization. In NeurIPS 2023 Foundation Models for Decision Making Workshop, 2023. 1

    2023

  41. [49]

    Expel: Llm agents are ex- periential learners

    Andrew Zhao, Daniel Huang, Quentin Xu, Matthieu Lin, Yong-Jin Liu, and Gao Huang. Expel: Llm agents are ex- periential learners. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 19632–19642, 2024. 2

    2024

  42. [50]

    Can gpt-4 perform neural architecture search? arXiv preprint arXiv:2304.10970, 2023

    Mingkai Zheng, Xiu Su, Shan You, Fei Wang, Chen Qian, Chang Xu, and Samuel Albanie. Can gpt-4 perform neural architecture search? arXiv preprint arXiv:2304.10970, 2023. 1, 2, 7

    2023 arXiv

  43. [51]

    Neural architecture search with reinforcement learning

    Barret Zoph and Quoc V Le. Neural architecture search with reinforcement learning. arXiv preprint arXiv:1611.01578 ,

    arXiv

  44. [53]

    Read and understand the title and abstract of the paper

  45. [54]

    Summarizing the innovation and contribution of this paper

  46. [55]

    ###Title### {{title}} ###Abstract### {{abstract}} ###Output### Answer yes or no prefix with ##response## in the end

    Analyzing whether you can get inspiration from this paper to design a better basic block architecture for visual models. ###Title### {{title}} ###Abstract### {{abstract}} ###Output### Answer yes or no prefix with ##response## in the end. A2.4. Reflector Reflector reflects on t...

  47. [56]

    Please focus on the methods of the paper to respond

    A paper usually contains several sections: abstract, introduction, related work, methods, experiments and conclusion. Please focus on the methods of the paper to respond

  48. [57]

    Inspirations must to be detailed and related to designing the basic block architecture of the visual model backbone. ###Input### The following is the content of the paper: {{paper}} ###Output### You response should wrap each inspirations with <inspiration> and </inspiration>, ...

  49. [58]

    1530 [39] 1

    Implement a Global Attention Block within the Task Net to extract task-specific features, focusing on both channel and spatial attention to capture relevant features for each task. 1530 [39] 1. Adoption of a multi-scale architecture using reversible residual blocks and squeeze...

  50. [59]

    Exclusion of normalization layers in the reversible blocks to facilitate learning direct style representation without interference, enhancing the style transfer fidelity

  51. [60]

    ##blockname##

    Implementing channel refinement in reversible residual blocks to manage redundant information accumu- lation and enhance stylization quality. Table A5. Prompt template for selecting modification suggestions. ###Instruction### You are a computer vision research expert, and you ...

  52. [61]

    Please ensure that the number of input channels and output channels of the generated block are both C

  53. [62]

    Note that structures in the modified block that unrelated to the proposal should be kept as original as possible

  54. [63]

    The new neural architecture you output must comply with the BlockDefinition format. ###BlockDefinition### {{Definition of the graph-based neural architecture representation}} ###block### {{block}} ###inspiration### {{inspiration}} ###Design Experience### Refer to the following...

  55. [64]

    The suggestion must be relevant to the neural network structure

  56. [65]

    The suggestion must be a sentence no more than 50 words

  57. [66]

    The suggestion must be must be general. ###Output### Please think step by step about the reasons why the accuracy of the model decreases after modification and give a suggestion to avoid this error in the next modification. The suggestion should be wrapped in<suggestion> and <...

  58. [67]

    You can achieve this by using the ‘permute’ operation to rearrange the tensor dimensions appropriately before applying LN

    When using LN, always ensure that the last dimension of the input tensor is the channel dimension. You can achieve this by using the ‘permute’ operation to rearrange the tensor dimensions appropriately before applying LN

  59. [68]

    This is necessary to maintain valid configurations and avoid errors

    When designing a neural network, ensure that the ‘outchannels’ parameter in the ‘Conv2d’ operation is divisible by the ‘groups’ parameter. This is necessary to maintain valid configurations and avoid errors. Specifically, if you set ‘groups‘ to be equal to ‘C’ (the number of i...

  60. [69]

    When designing a neural network with multiple branches that will be concatenated or added together, ensure that all branches maintain consistent spatial dimensions (height and width) throughout their respective operations. Use padding in convolution and pooling layers to prese...

  61. [70]

    Ensure modifications retain the original block’s dimensional consistency and spatial information to maintain performance

  62. [71]

    Ensure that added layers and operations contribute to meaningful feature extraction and avoid unnecessary complexity

  63. [72]

    Consider maintaining intermediate bottleneck layers and incorporating attention or pooling mechanisms to enhance fea- ture extraction

  64. [73]

    Ensure that any added skip connections or operations do not disrupt the learning process by excessively altering the network’s expected data flow

  65. [74]

    Ensure the newly added paths or operations do not interfere destructively with existing paths, and validate their impact on gradient flow. 15 A3. Detailed Experimental Results A3.1. Detailed Numberical Results Table A12, Table A13, and Table A14 show the experimen- tal results...

  66. [2016]

    Detailed Experimental Setting A1.1

    1 10 A1. Detailed Experimental Setting A1.1. Macro Skeleton When conducting experiments on CIFAR10, CI- FAR100 [18], and ImageNet16-120 [7], in order to compare fairly with the compared methods, we constrain the model designed by our methods to have the same macro skeleton as ...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.