REVIEW 4 major objections 7 minor 27 references
A four-agent LLM pipeline can automate CGRA hardware/software co-design, cutting power by 59% and producing a working 7nm layout.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-04 16:27 UTC pith:UZNBG3C2
load-bearing objection Real framework, one interesting mechanism, but the abstract and the body contradict each other on the headline results, and the body itself undercuts the 5x claim. the 4 major comments →
MACO: A Multi-Agent LLM Framework for Automated CGRA Hardware/Software Co-Design
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The paper's central claim is that a multi-agent LLM framework can automate CGRA hardware/software co-design and beat both manual design and simpler LLM prompting on power. The body reports average power reductions of 59.4% versus Qwen few-shot, 48.9% versus domain-augmented Qwen, and 27.1% versus manual design, plus a 36.8% power-efficiency gain; one generated 4x4 CGRA completed a 7nm ASIC flow at 2.99 mm², 30.49 mW, and 435 MHz. The abstract calls the system MACO and claims 25.9% power reduction, 20.0% performance improvement, and 5x search acceleration, while the body's Table II says early iterations take longer because tools are called multiple times. The key idea is that EDA feedback tea
What carries the argument
The central mechanism is Algorithm 1, an adaptive-confidence selector: it computes exp(-|l_score - t_score|/sigma) between the LLM's ranking and the EDA tool's ranking, updates a confidence estimate by exponential moving average, and once confidence exceeds a threshold it lets the LLM choose without tool evaluation, with periodic forced validation. The surrounding pipeline is organized as four collaborating agents: co-design, error correction, multi-judge selection, and evaluation/feedback. This confidence-switching procedure is what converts tool-verified iterations into fast LLM-only iterations.
Load-bearing premise
The framework's speedup rests on the assumption that the exponential similarity between LLM scores and EDA tool scores in Algorithm 1 is a dependable predictor of the LLM's selection accuracy once confidence passes the threshold; the paper specifies neither the score scales nor sigma, and after the switch the LLM chooses without tool verification.
What would settle it
Silently run the EDA tool for, say, 10 more iterations after the confidence threshold is crossed and compare the LLM's choices with the tool's choices on the same candidate sets; if agreement is not near-perfect, the self-learning selector is not yet trustworthy, and the reported 5x speedup should be re-benchmarked against always-tool selection including periodic validation calls.
If this is right
- CGRA design for a target kernel can be carried out end-to-end by the four-agent loop, from candidate generation through RTL and physical design, without manual parameter tuning.
- Replacing tool-based selection with LLM selection after confidence matures is what enables faster iteration; the paper reports 5x search acceleration.
- Jointly optimizing hardware and software parameters yields better power efficiency than optimizing hardware alone, as shown by the ablation without software optimization.
- The generated designs are not just functional; at least one MALTA-generated 4x4 CGRA completed a 7nm ASIC layout at 30.49 mW and 435 MHz.
Where Pith is reading between the lines
- The abstract's headline numbers (25.9% power, 20.0% performance, 5x search acceleration) do not match the body's reported averages (59.4%, 48.9%, 27.1% power reductions; 36.8% power-efficiency gain), and the body's Table II notes that runtime is longer in early iterations; the 5x acceleration claim needs a consistent accounting of when tool calls are counted.
- The confidence-switching mechanism is where the speedup comes from; a straightforward test is to log the LLM's chosen design versus the tool's chosen design after confidence crosses the threshold. If the agreement rate stays near chance, the acceleration is buying speed at the cost of unvalidated selections.
- Since sigma and the score scales in exp(-|l_score - t_score|/sigma) are not specified, the confidence values are not reproducible across models and tools; calibrating sigma per model would be a concrete extension.
- The current evaluation is per-kernel optimization; a natural next test is multi-kernel or domain-level co-design where one CGRA must serve several kernels, which would stress the selection stage differently than single-kernel power minimization.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a multi-agent LLM framework for coarse-grained reconfigurable array (CGRA) hardware/software co-design. The body calls the framework MALTA, whereas the abstract refers to MACO. The framework decomposes the design loop into four stages: CGRA co-design, validation/correction, best-design selection, and evaluation/feedback. It integrates an OpenCGRA-based mapper (MapFixer), RTL generation, and Synopsys Design Compiler for power/area estimates, and introduces an adaptive-confidence mechanism in which an LLM gradually replaces EDA-tool-based selection after its choices become sufficiently consistent with tool reports. The evaluation covers 8 kernels across three domains, comparing against Qwen few-shot, Qwen domain-augmented, and manual design. The body reports power reductions of 59.4%, 48.9%, and 27.1% versus those baselines, a 36.8% power-efficiency improvement, and one 4x4 CGRA taken through an ASAP7 7nm ASIC flow (2.99 mm², 30.49 mW, 435 MHz). The abstract, however, claims different headline numbers: 25.9% power reduction, 20.0% performance improvement, 5x search acceleration, and validation through a complete 7nm flow.
Significance. Automating CGRA HW/SW co-design with LLM agents is a timely and useful goal, and the body demonstrates a working system that can generate and correct CGRA designs, integrate external EDA tools, and carry one design through physical implementation. The open-source claim, if backed by a released repository, would be a community asset. The reported per-kernel power/efficiency results in Table II and Figure 4 do support the weaker claim that MALTA often finds lower-power CGRAs than the tested baselines under a speedup constraint. However, the significance is substantially undermined by the abstract/body mismatch: the abstract's headline numbers are not present in the body, and the one claimed acceleration result (5x) is contradicted by the runtime data in Table II. The self-learning mechanism that would enable acceleration is never validated. Thus, the paper's central quantitative claims are not currently supported by the evidence.
major comments (4)
- [Abstract vs. §IV-B, Table II] The abstract's headline claims (25.9% power reduction, 20.0% performance improvement, 5x search acceleration) do not appear in the body. The body reports different numbers: power reductions of 59.4%, 48.9%, and 27.1% versus the three baselines, plus a 36.8% power-efficiency improvement. No 20% performance improvement appears anywhere, and 'performance' is never defined. More seriously, Table II's runtime column shows MALTA's runtime is larger than every baseline for every kernel (e.g., fir: 185.6 min vs 104.1–137.6; fft: 252.8 vs 110.2–132.8; spmv: 150.4 vs 89.8–117.9; gemm: 216.3 vs 100.7–144.8), and the table note states: 'Since our framework calls the tool multiple times in the early stages, it tends to consume a longer time.' This directly contradicts the abstract's 'accelerates the search process by 5x.' The central quantitative claims are therefore neither verifiable from the body
- [§III-C, Algorithm 1] The adaptive-confidence self-learning mechanism is the only proposed route to accelerating selection, but it is never tested. The similarity exp(−|l_score−t_score|/σ) uses an unspecified σ and undefined score scales. Conf_Threshold, Validation_Interval N, and EMA factor α are not given. No experiment reports the number of tool evaluations skipped, the confidence trajectory, or the accuracy of LLM-only selections after switching to LLM-only mode. Thus, the claim that this mechanism accelerates the design process is unsupported. A minimal validation would compare wall-clock time or tool-call counts with and without the self-learning switch, and plot selection accuracy against the confidence threshold.
- [§IV-A, §IV-B, Figure 4] The performance metric is not defined. The abstract claims a 20% 'performance improvement,' but the body only reports speedup relative to an in-order CPU and power efficiency, without defining whether performance means throughput, latency, cycles, or something else. Figure 4's axis is labeled 'Normalized Power Efficiency' but the normalization baseline and the definition of power efficiency are not stated. The absence of a defined metric makes the reported improvements non-reproducible. Additionally, the number of independent runs and run-to-run variance are not reported for the 10-iteration results.
- [§IV-B, Physical Implementation] The claim of validating physical design through a 'complete 7nm ASIC design flow' rests on a single 4x4 CGRA generated for the embedded domain, with area 2.99 mm², power 30.49 mW, and frequency 435 MHz. There is no comparison baseline, no timing/power analysis against an alternative design, and no evidence that this design is the result of MALTA's iterative optimization rather than a single generated candidate. This demonstrates that one MALTA-generated RTL can be passed through ASAP7; it does not validate the framework's design quality or physical-design awareness. The abstract's wording overstates this evidence.
minor comments (7)
- [Throughout] The framework is called MACO in the abstract and MALTA in the body, including the title of the full text. This naming inconsistency must be resolved.
- [Algorithm 1] Variables t_choice, t_score, l_choice, and l_score are used before being explicitly defined; clarify their roles and units.
- [Table III] The table is described as 'one-shot' while the framework is iterative; clarify how one-shot relates to the 10-iteration results in Table II.
- [Table IV] Table IV lists HW/SW parameters for one kernel (spmv) but provides no associated power/performance results; either add the measurements or point explicitly to Figure 4.
- [Figure 5] Convergence curves for fft, spmv, and gemm show no error bars or confidence intervals; specify whether these are best-of-10, median, or mean values.
- [Abstract / §IV] The abstract claims the framework is 'open-source,' but no repository link or artifact availability statement is provided in the manuscript.
- [General] There are multiple typos and stylistic issues, e.g., 'Framwork', 'effiencicy', and 'none HW/SW co-design approaches'; a careful proofread is needed.
Circularity Check
No significant circularity; the framework's claims rest on external tool evaluations and calibration against tool reports, not on self-defined or fitted targets.
full rationale
The paper's derivation chain is not circular. MALTA is an iterative search loop in which candidate CGRA designs are validated and scored by external tools: OpenCGRA [26] for mapping/speedup, Synopsys Design Compiler for power/area, and ASAP7 for physical design. The reported power reductions, speedups, and area values are direct measurements from these tools, not quantities derived from the framework's own definitions. The LLM self-learning mechanism in Algorithm 1 computes a confidence score as an exponential moving average of the similarity between LLM scores and tool scores; this is a calibration/surrogate-model approach, and the confidence threshold only decides when to skip tool evaluation. It does not redefine the evaluation metric nor fit the final power/performance numbers. The only provenance concern is that OpenCGRA [26] is authored by a co-author (Cheng Tan) and is used both as the design-validation backend and the speedup evaluation baseline. However, OpenCGRA is an open-source external tool, so this is a minor self-citation rather than load-bearing circularity; it does not make the central multi-agent co-design contribution reduce to its own input. The abstract contains numbers (25.9% power, 20% performance, 5x search acceleration) that do not appear in the body, and Table II's runtime column explicitly shows MALTA taking longer than baselines, contradicting the 5x acceleration claim. This is an internal-consistency/correctness problem, not a circular-derivation problem, so it is noted but does not raise the circularity score.
Axiom & Free-Parameter Ledger
free parameters (5)
- sigma (similarity scale) =
not specified
- Conf_Threshold =
not specified
- Validation_Interval (N) =
not specified
- alpha (EMA factor) =
not specified
- Number of candidate designs K =
not specified
axioms (5)
- domain assumption MapFixer and rule-based error correction can repair all structural and compilation errors in LLM-generated CGRA designs
- domain assumption OpenCGRA toolflow [26] faithfully models speedup and power of generated CGRAs
- domain assumption EMA confidence from LLM/tool score agreement generalizes to future iterations
- domain assumption LLM textual reports can transfer knowledge across iterations
- domain assumption Designs from 10 iterations converge to a good optimum
read the original abstract
Designing optimal Coarse-Grained Reconfigurable Arrays (CGRAs) requires navigating a vast, interdependent hardware/software space bottlenecked by costly manual iteration. We present MACO, an open-source, multi-agent LLM framework that automates CGRA HW/SW co-design. MACO decomposes the design loop into four collaborative stages, HW/SW Co-design, Error Correction, Best-Design Selection, and Evaluation & Feedback, to iteratively optimize power, performance, and area (PPA). To accelerate convergence and efficiently traverse the design space, MACO introduces an exponentially decaying exploration strategy, EDA-guided LLM self-learning, and robust rule-based error correction. Evaluated against state-of-the-art baselines, MACO reduces power consumption by 25.9%, improves performance by 20.0%, and accelerates the search process by 5x. Finally, we validate MACO's physical design through a complete 7nm ASIC design flow.
Figures
Reference graph
Works this paper leans on
-
[1]
Wenji Fang, Jing Wang, Yao Lu, Shang Liu, Yuchao Wu, Yuzhe Ma, and Zhiyao Xie. A survey of circuit foundation model: Foundation ai models for vlsi circuit design and eda.arXiv preprint arXiv:2504.03711, 2025
Pith/arXiv arXiv 2025
-
[2]
Automated cpu design by learning from input-output examples
Shuyao Cheng, Pengwei Jin, Qi Guo, Zidong Du, Rui Zhang, Xing Hu, Yongwei Zhao, Yifan Hao, Xiangtao Guan, Husheng Han, et al. Automated cpu design by learning from input-output examples. In Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, pages 3843–3853, 2024
2024
-
[3]
Chatcpu: An agile cpu design and verification platform with llm
Xi Wang, Gwok-Waa Wan, Sam-Zaak Wong, Layton Zhang, Tianyang Liu, Qi Tian, and Jianmin Ye. Chatcpu: An agile cpu design and verification platform with llm. InProceedings of the 61st ACM/IEEE Design Automation Conference, pages 1–6, 2024
2024
-
[4]
Nicolas Dupuis, Ravi Nair, Shyam Ramji, Sean McClintock, Nishant Chauhan, Priyanka Nagpal, Bart Blaner, Ken Valk, Leon Stok, and Ruchir Puri. Customizing a large language model for vhdl design of high-performance microprocessors.arXiv preprint arXiv:2505.09610, 2025
Pith/arXiv arXiv 2025
-
[5]
Large processor chip model.arXiv preprint arXiv:2506.02929, 2025
Kaiyan Chang, Mingzhi Chen, Yunji Chen, Zhirong Chen, Dongrui Fan, Junfeng Gong, Nan Guo, Yinhe Han, Qinfen Hao, Shuo Hou, et al. Large processor chip model.arXiv preprint arXiv:2506.02929, 2025
Pith/arXiv arXiv 2025
-
[6]
Tpu- gen: Llm-driven custom tensor processing unit generator.arXiv preprint arXiv:2503.05951, 2025
Deepak Vungarala, Mohammed E Elbtity, Sumiya Syed, Sakila Alam, Kartik Pandit, Arnob Ghosh, Ramtin Zand, and Shaahin Angizi. Tpu- gen: Llm-driven custom tensor processing unit generator.arXiv preprint arXiv:2503.05951, 2025
Pith/arXiv arXiv 2025
-
[7]
Gpt4aigchip: Towards next-generation ai accelerator design automation via large language models
Yonggan Fu, Yongan Zhang, Zhongzhi Yu, Sixu Li, Zhifan Ye, Chaojian Li, Cheng Wan, and Yingyan Celine Lin. Gpt4aigchip: Towards next-generation ai accelerator design automation via large language models. In2023 IEEE/ACM International Conference on Computer Aided Design (ICCAD), pages 1–9. IEEE, 2023
2023
-
[8]
Fedchip: Federated llm for artificial intelligence accelerator chip design
Mahmoud Nazzal, Khoa Nguyen, Deepak Vungarala, Ramtin Zand, Shaahin Angizi, Hai Phan, and Abdallah Khreishah. Fedchip: Federated llm for artificial intelligence accelerator chip design. In2025 IEEE International Conference on LLM-Aided Design (ICLAD), pages 93–99. IEEE, 2025
2025
-
[9]
Zeju Li, Changran Xu, Zhengyuan Shi, Zedong Peng, Yi Liu, Yun- hao Zhou, Lingfeng Zhou, Chengyu Ma, Jianyuan Zhong, Xi Wang, et al. Deepcircuitx: A comprehensive repository-level dataset for rtl code understanding, generation, and ppa analysis.arXiv preprint arXiv:2502.18297, 2025
Pith/arXiv arXiv 2025
-
[10]
Mg-verilog: Multi-grained dataset towards enhanced llm- assisted verilog generation
Yongan Zhang, Zhongzhi Yu, Yonggan Fu, Cheng Wan, and Yingyan Ce- line Lin. Mg-verilog: Multi-grained dataset towards enhanced llm- assisted verilog generation. In2024 IEEE LLM Aided Design Workshop (LAD), pages 1–5. IEEE, 2024
2024
-
[11]
Sa-ds: A dataset for large language model-driven ai accelerator design generation
Deepak Vungarala, Mahmoud Nazzal, Mehrdad Morsali, Chao Zhang, Arnob Ghosh, Abdallah Khreishah, and Shaahin Angizi. Sa-ds: A dataset for large language model-driven ai accelerator design generation. In 2025 IEEE International Symposium on Circuits and Systems (ISCAS), pages 1–4. IEEE, 2025
2025
-
[12]
Manar Abdelatty, Jacob Rosenstein, and Sherief Reda. Hdlcopilot: Natural language exploration of hardware designs and libraries.arXiv preprint arXiv:2407.12749, 2024
Pith/arXiv arXiv 2024
-
[13]
Verilogcoder: Autonomous verilog coding agents with graph-based planning and abstract syntax tree (ast)-based waveform tracing tool
Chia-Tung Ho, Haoxing Ren, and Brucek Khailany. Verilogcoder: Autonomous verilog coding agents with graph-based planning and abstract syntax tree (ast)-based waveform tracing tool. InProceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 300– 307, 2025
2025
-
[14]
Verigen: A large language model for verilog code generation.ACM Transactions on Design Automation of Electronic Systems, 29(3):1–31, 2024
Shailja Thakur, Baleegh Ahmad, Hammond Pearce, Benjamin Tan, Brendan Dolan-Gavitt, Ramesh Karri, and Siddharth Garg. Verigen: A large language model for verilog code generation.ACM Transactions on Design Automation of Electronic Systems, 29(3):1–31, 2024
2024
-
[15]
Autochip: Automating hdl generation using llm feedback.arXiv preprint arXiv:2311.04887, 2023
Shailja Thakur, Jason Blocklove, Hammond Pearce, Benjamin Tan, Sid- dharth Garg, and Ramesh Karri. Autochip: Automating hdl generation using llm feedback.arXiv preprint arXiv:2311.04887, 2023
Pith/arXiv arXiv 2023
-
[16]
Rtllm: An open- source benchmark for design rtl generation with large language model
Yao Lu, Shang Liu, Qijun Zhang, and Zhiyao Xie. Rtllm: An open- source benchmark for design rtl generation with large language model. In2024 29th Asia and South Pacific Design Automation Conference (ASP-DAC), pages 722–727. IEEE, 2024
2024
-
[17]
Yi Liu, Changran Xu, Yunhao Zhou, Zeju Li, and Qiang Xu. Deeprtl: Bridging verilog understanding and generation with a unified represen- tation model.arXiv preprint arXiv:2502.15832, 2025
Pith/arXiv arXiv 2025
-
[18]
Shang Liu, Wenji Fang, Yao Lu, Qijun Zhang, Hongce Zhang, and Zhiyao Xie. Rtlcoder: Outperforming gpt-3.5 in design rtl generation with our open-source dataset and lightweight solution.arXiv preprint arXiv:2312.08617, 2023
Pith/arXiv arXiv 2023
-
[19]
Jason Blocklove, Siddharth Garg, Ramesh Karri, and Hammond Pearce. Chip-chat: Challenges and opportunities in conversational hardware design.arXiv preprint arXiv:2305.13243, 2023
Pith/arXiv arXiv 2023
-
[20]
Natural language is not enough: Benchmarking multi-modal generative ai for verilog generation
Kaiyan Chang, Zhirong Chen, Yunhao Zhou, Wenlong Zhu, Kun Wang, Haobo Xu, Cangyuan Li, Mengdi Wang, Shengwen Liang, Huawei Li, et al. Natural language is not enough: Benchmarking multi-modal generative ai for verilog generation. InProceedings of the 43rd IEEE/ACM International Conference on Computer-Aided Design, pages 1–9, 2024
2024
-
[21]
Origen: Enhancing rtl code generation with code-to-code aug- mentation and self-reflection
Fan Cui, Chenyang Yin, Kexing Zhou, Youwei Xiao, Guangyu Sun, Qiang Xu, Qipeng Guo, Yun Liang, Xingcheng Zhang, Demin Song, et al. Origen: Enhancing rtl code generation with code-to-code aug- mentation and self-reflection. InProceedings of the 43rd IEEE/ACM International Conference on Computer-Aided Design, pages 1–9, 2024
2024
-
[22]
Rtl++: Graph- enhanced llm for rtl code generation.arXiv preprint arXiv:2505.13479, 2025
Mohammad Akyash, Kimia Azar, and Hadi Kamali. Rtl++: Graph- enhanced llm for rtl code generation.arXiv preprint arXiv:2505.13479, 2025
Pith/arXiv arXiv 2025
-
[23]
A data-centric chip design agent framework for verilog code generation
Kaiyan Chang, Wenlong Zhu, Kun Wang, Xinyang He, Nan Yang, Zhirong Chen, Dantong Jin, Cangyuan Li, Yunhao Zhou, Hao Yan, et al. A data-centric chip design agent framework for verilog code generation. ACM Transactions on Design Automation of Electronic Systems, 2025
2025
-
[24]
Chipnemo: Domain-adapted llms for chip design.arXiv preprint arXiv:2311.00176, 2023
Mingjie Liu, Teodor-Dumitru Ene, Robert Kirby, Chris Cheng, Nathaniel Pinckney, Rongjian Liang, Jonah Alben, Himyanshu Anand, Sanmitra Banerjee, Ismet Bayraktaroglu, et al. Chipnemo: Domain-adapted llms for chip design.arXiv preprint arXiv:2311.00176, 2023
Pith/arXiv arXiv 2023
-
[25]
Ning Wang, Bingkun Yao, Jie Zhou, Xi Wang, Zhe Jiang, and Nan Guan. Large language model for verilog generation with code-structure-guided reinforcement learning.arXiv preprint arXiv:2407.18271, 2024
Pith/arXiv arXiv 2024
-
[26]
Opencgra: An open-source unified framework for modeling, testing, and evaluating cgras
Cheng Tan, Chenhao Xie, Ang Li, Kevin J Barker, and Antonino Tumeo. Opencgra: An open-source unified framework for modeling, testing, and evaluating cgras. In2020 IEEE 38th International Conference on Computer Design (ICCD), pages 381–388. IEEE, 2020
2020
-
[27]
Asap7: A 7-nm finfet predictive process design kit.Microelec- tronics Journal, 53:105–115, 2016
Lawrence T Clark, Vinay Vashishtha, Lucian Shifren, Aditya Gujja, Saurabh Sinha, Brian Cline, Chandarasekaran Ramamurthy, and Greg Yeric. Asap7: A 7-nm finfet predictive process design kit.Microelec- tronics Journal, 53:105–115, 2016. 7
2016
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.