Pith. sign in

REVIEW 3 major objections 7 minor 49 references

Enabling New HDLs with Agents

T0 review · 3 major / 7 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Wrapping an off-the-shelf LLM with a distilled HDL summary, few-shot examples, and iterative compiler feedback lifts its success on chip languages that were absent from training from near zero to 35-59%, and above 90% for small snippets.

desk verdict A credible low-resource-HDL coding agent with a genuinely informative ablation, but the language summaries were tuned on the same benchmark used for the headline numbers, so the gains are likely optimistic until held-out validation appears. read the letter →

arxiv 2501.00642 v1 pith:FHIAKPY7 submitted 2024-12-31 cs.AR cs.AIcs.LGcs.PL

Mark Zakharov , Farzaneh Rabiei Kashanaki , Jose Renau This is my paper · ORCID
classification cs.ARcs.AIcs.LGcs.PL
keywords HDLAgentlargelanguagemodelshardwaredescriptionlanguagescodegenerationcompilerfeedbackfew-shotlearningtransferChisel
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that off-the-shelf large language models can be made productive for hardware description languages (HDLs) that were missing from their training data, without fine-tuning or retraining. The authors introduce HDLAgent, a prompt-and-iteration wrapper that supplies the model with a compact summary of the target HDL, a few short worked examples, and a loop that feeds compiler error messages back until the code compiles. On the HDLEval and VerilogEval benchmarks, the agent raised PyRTL success from 0% to 35% with Mixtral 8x7B and Chisel from 0% to 59% with GPT-3.5-turbo-0125, and it pushed success above 90% for small snippets across all tested HDLs. The authors read this as evidence that new HDLs can ride on existing LLM knowledge, lowering a barrier to adoption that would otherwise make LLMs a brake on hardware-language innovation.

What carries the argument

The load-bearing object is the two-part context that HDLAgent feeds to the LLM at each step. The main context contains an LLM-generated summary of the target HDL, chosen to be concise and to emphasize syntax that differs from Verilog, a fixed set of few-shot examples for common constructs, and prefix and suffix statements that pin down input-output conventions and forbid English prose. The compiler context is the iterative loop: after a failed compile, the agent sends the whole previous code snippet with the compiler error message, plus an optional generic sample fix, and asks for a corrected program; a per-language filter strips non-code text and locates code boundaries. The paper attributes the gains to transfer learning in the LLM, with the summary acting as a bridge from Verilog knowledge to the new HDL.

What would settle it

Run HDLAgent on a brand-new HDL, or on held-out problems chosen after the context is fixed, using summaries and examples created without ever seeing the test suite; if success rates stay near the no-agent baseline for that language, the reported gains are benchmark-specific rather than evidence of transferable capability.

Watch

Extended reading notes

Core claim

The central claim is that the gap between LLMs and emerging HDLs is not a training-data problem that only fine-tuning can fix; a generic agent recipe can close much of it at inference time. HDLAgent composes two memory blocks: a main context that distills the HDL's reference manual into a short description plus few-shot examples of tricky syntax such as bit operations, reductions, loops, multiplexing, and multiply-add blocks, and a compiler context that, on a compile failure, reintroduces the entire latest code snippet with the raw compiler error message and, when available, a sample fix. With up to eight iterations, this grounded loop moves success rates from near zero for all six tested LLMs on Chisel, PyRTL, and DSLX to a range where even the weakest model reaches 28% on PyRTL and 53% on Verilog, and where small examples exceed 90% on every HDL. The paper also reports that feeding the full reference manual is worse than a summary, and that for DSL-based HDLs like Chisel and PyRTL the combination of description plus compiler feedback matters more than either alone.

Load-bearing premise

The paper's transferable gains rest on the HDL summaries and example snippets being generic descriptions of each language, but the authors pick the summary that scores highest on the evaluation models, so if those summaries encode knowledge of the test problems, the reported increases will not generalize to an unseen HDL.

Editorial extensions

If this is right

  • New HDLs can become usable by off-the-shelf LLMs immediately, without waiting for fine-tuning datasets or closed-model retraining cycles.
  • For snippets under roughly 25 lines of equivalent Verilog, HDLAgent reaches over 90% success on Chisel, PyRTL, DSLX, and Verilog across the tested LLMs, which the paper argues is enough for learners asking for small code examples.
  • Because success still collapses on modules beyond about 75 lines, the paper identifies large-design generation as the open problem that agent wrappers alone do not solve.
  • The authors propose concrete HDL and compiler design changes, such as clearer error messages that distinguish DSL syntax from host-language syntax and multi-output port conventions in DSLX, that would reduce the number of iterations HDLAgent needs.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the context-construction step is fully automated by having an LLM summarize any language manual with the same generic prompt, the recipe may transfer to non-hardware DSLs and other niche programming languages.
  • The reported gains are likely optimistic for a truly novel HDL, because the paper selects the summary that scores best on the evaluation LLMs; a blind test on a fresh HDL would be needed to confirm how much transfer is real.
  • The observation that summaries beat full manuals suggests that difference-focused documentation, not exhaustive references, is what limited-context models need, which could inform how new languages write their own onboarding guides.
  • Combining HDLAgent with a small fine-tuned repair model, in the spirit of existing debugger agents, is a natural next step toward closing the large-design gap the paper documents.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. This paper proposes HDLAgent, an agentic prompting framework that wraps off-the-shelf LLMs to generate code in HDLs underrepresented in training data (Chisel, PyRTL, DSLX) as well as Verilog. The framework has a 'main context' consisting of an LLM-generated HDL summary, few-shot examples, and prompt prefix/suffix, and a 'compiler context' that iterates up to eight times on compiler errors, optionally with sample fixes. The evaluation uses HDLEval and VerilogEval across six LLMs and reports large improvements; for example, Table 3 shows GPT-3.5n on Chisel rising from 0/134 to 80/134 and Mix-8x7B on PyRTL rising from 0/134 to 48/134 on HDLEval-Comb. The paper also analyzes pass@k, iteration counts, token cost, execution time, QoR, and derives HDL-specific design recommendations.

Significance. If the reported gains hold outside the specific benchmark, HDLAgent would be a practical, fine-tuning-free method for making LLMs usable for niche HDLs, with implications for tool adoption and for HDL designers. The paper's strengths include breadth across LLMs and HDLs, an explicit component ablation, top@k and token-cost analysis, QoR measurements, and honest discussion of failure modes. The main caveat is that the main-context components were selected using the same HDLEval benchmark on which the headline numbers are reported; therefore external validity needs to be established before the central claim can be fully credited.

major comments (3)
  1. [Section 3.1, Section 5.2, Figure 7, Table 3] The paper states in Section 3.1 that 'we opt for simplicity by selecting the description that performs best with Mix-8x7B and GPT-3.5n,' and Section 5.2/Figure 7 ranks summarization prompts by accuracy on HDLEval. Because the headline gains in Table 3 and the abstract are computed on the same HDLEval data, the main-context prompt has effectively been selected on the evaluation set. The paper needs a genuine held-out evaluation (e.g., a development/test split of HDLEval made before prompt selection, or an independent set of tasks in each new HDL) before the central claim that HDLAgent transfers to unseen HDL programs is supported. Without this, the reported 0-to-60% improvements could be benchmark-specific rather than evidence of general enablement of new HDLs.
  2. [Section 3.2, Figures 3-6] The 'sample fix' mechanism is a load-bearing component: the Fixes rows in Figures 3-6 show substantial additional gains beyond the Compile step, and Section 3.2 says HDLAgent appends a sample fix when one exists. However, the paper never states where these sample fixes come from, how many are used, or whether they were harvested from the HDLEval problem set. This is a second potential leakage channel in addition to the main-context selection. Please specify the provenance of sample fixes, provide the actual fixes in an appendix, and confirm that they are generic and independent of the evaluation problems.
  3. [Conclusion, Section 5.1, Table 3] The conclusion states that HDLAgent raised the Verilog success rate of GPT-4 from 34% to 72%. I could not locate a 34% baseline in the body: Table 3 reports GPT-4 with Verilog on HDLEval-Comb at 97/134 (72.4%) Base and 102/134 (76.1%) HDLAgent at k=1. If the 34% figure comes from a different benchmark or subset, it should be identified explicitly; otherwise the conclusion is inconsistent with the paper's own data.
minor comments (7)
  1. [Section 5.1] The subsection heading 'PyR TL' contains a typo; it should read 'PyRTL'.
  2. [Listing 2] The code sample contains 'inpput' (misspelling of 'input'); if this is literal code, it may confuse readers trying to reproduce the example.
  3. [Table 3] Table 3 entries appear to be numbers of tests passed out of 134 HDLEval-Comb tests, but the table does not state the denominator; please state this directly so readers can convert to percentages.
  4. [Section 5.5, Figure 11] The claim of over 90% success on concise examples is based on Figure 11, which uses GPT-4 only; please qualify the claim by LLM and by benchmark condition.
  5. [Section 3.3] The paper mentions a per-language filter/detector for removing English and finding code boundaries but gives only the Verilog example; please specify the filter rules for Chisel, PyRTL, and DSLX for reproducibility.
  6. [References, Section 3.3] Reference [14] (FinanceBench) does not appear to support the claim that placing the prompt after the context achieves better results; please provide an appropriate citation.
  7. [Section 5.2] The comparison between a full reference manual and a summary reports percentages (e.g., GPT-4 from 77% to 66%, Mix-8x7B from 59% to 33%) without tying them to a specific table or benchmark condition; please clarify the conditions for these numbers.

Circularity Check

1 steps flagged · score 4.0 of 10

Main context is tuned on the evaluation benchmark, so reported gains are partly test-set selection; external VerilogEval results keep the core claim independently grounded.

  1. fitted input called prediction [Section 3.1 (Main Context), with Figure 7 and Section 5.2 (HDLAgent Context Insights)]
    "While our evaluation demonstrates that the HDL description can be optimized for each LLM, we opt for simplicity by selecting the description that performs best with Mix-8x7B and GPT-3.5n."

    The headline success rates (e.g., PyRTL 0 to 35%, Chisel 0 to 59% in the abstract; Table 3) are computed on the HDLEval benchmark, which is the same benchmark used to select the HDL description and few-shot context. Figure 7 explicitly ranks summarization-prompt variants by their HDLEval success rate, and Section 5.2 reports that different options were compared. Thus the reported numbers are the result of choosing the best-performing context on the evaluation set, then re-measuring on that same set.

full rationale

The core mechanism of HDLAgent — a Verilog-oriented description plus few-shot examples plus compiler-feedback iteration — is not itself defined in terms of the benchmark outcomes, and the compiler-context loop is a general technique with external precedent. The main circularity burden is the prompt-selection step: Section 3.1 admits choosing the HDL description that performs best on the evaluation LLMs, and Figure 7 shows that choice was made by comparing success rates on HDLEval, the same corpus used for the article's headline numbers. This is test-set tuning rather than a derivation-by-definition, so it does not make the whole result forced; however, it does mean the reported gains are benchmark-specific maxima over prompt variants rather than independent predictions. The paper's use of the self-authored HDLEval benchmark is mitigated because HDLEval is derived from external HDLBits and Efabless tasks, and the external VerilogEval results and the consistent improvement across six LLMs provide independent support. For those reasons, the circularity is moderate (4/10) rather than total.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

No new physical entities or conceptual objects are postulated; HDLAgent is a software framework. The load-bearing assumptions are about benchmark representativeness, knowledge transfer from Verilog, and the genericity of the fix examples. The free parameters are the selected HDL description and the iteration limit, both chosen empirically.

free parameters (2)
  • HDL description selection = Summary generated by GPT-4 or GPro-1.0 that achieved the highest success rate on HDLEval with Mix-8x7B and GPT-3.5n
    Section 3.1: 'we opt for simplicity by selecting the description that performs best with Mix-8x7B and GPT-3.5n.' Choosing the best-performing context on the evaluation benchmark is a form of tuning to the test set.
  • Compiler iteration limit = 8
    Section 5.3: '8 iterations prove sufficient across languages, as increasing iterations further fails to improve success rates while incurring higher token usage.' Chosen empirically from Figure 8.
assumptions (3)
  • domain assumption HDLEval benchmark is a representative sample of real HDL coding tasks
    The evaluation focuses on HDLEval-Comb and HDLEval-Pipe, a benchmark introduced by the same authors [44] and derived from HDLBits and Efabless competition problems. Success on these tests is treated as evidence of general HDL code generation ability.
  • ad hoc to paper Verilog knowledge transfers to other HDLs through a summary and few-shot examples
    Section 3 states the agent 'exploits this capability, enabling LLMs to handle HDLs with limited training data,' but the extent of transfer is not independently verified beyond the selected benchmark.
  • ad hoc to paper Error-fix examples are generic and not tailored to the benchmark questions
    Section 3.2 describes appending a 'sample fix' for compiler errors, but the paper does not specify how these fixes are generated or whether they were chosen based on the benchmark tasks. If they are problem-specific, the success rates would not transfer.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Enabling New HDLs with Agents." pith.science (2026). https://pith.science/paper/FHIAKPY7

@misc{pith2026250100642,
  author       = {Pith},
  title        = {Pith review of: Enabling New HDLs with Agents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FHIAKPY7}},
  note         = {Machine review of arXiv:2501.00642}
}
read the original abstract

Large Language Models (LLMs) based agents are transforming the programming language landscape by facilitating learning for beginners, enabling code generation, and optimizing documentation workflows. Hardware Description Languages (HDLs), with their smaller user community, stand to benefit significantly from the application of LLMs as tools for learning new HDLs. This paper investigates the challenges and solutions of enabling LLMs for HDLs, particularly for HDLs that LLMs have not been previously trained on. This work introduces HDLAgent, an AI agent optimized for LLMs with limited knowledge of various HDLs. It significantly enhances off-the-shelf LLMs.

Figures

Figures reproduced from arXiv: 2501.00642 by the authors.

Figure 1
Figure 1. HDLAgent flow leveraging compiler feed￾back. As illustrated in [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. HDLAgent Main and Compiler context prompt components. 3.1 Main Context The "main context" in HDLAgent serves to inform the LLM about the specific HDL in use [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. HDLAgent improves Chisel across all LLMs. Both the "main context" (comprising Description and Few-shot components) and the "compiler context" (including Compile and Fixes elements) provide sub￾stantial benefits, underscoring the necessity of all these components. Notably, with HDLAgent, GPT￾3.5o and GPT-3.5n outperform even high-performing LLMs like GPT-4 in its baseline state. Further￾more, HDLAgent significantly e… view at source ↗
Figures from the paper (7 more)
Figure 5
Figure 5. Figure 5: HDLAgent improves DSLX HDLEval￾Comb across all LLMs. DSLX ( [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Verilog succeeds across benchmarks and LLMs . Verilog ( [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: HDL description and few-shot help LLMs to improve results. [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 9
Figure 9. Figure 9: LLM and HDL affect total HDLAgent ex￾ecution time. 5.4 Time and QoR Execution time is a crucial metric for any AI Agent. It refers to the time HDLAgent requires to generate a response, not the quality of results (QoR) [PITH_FULL_IMAGE:figures/full_fig_p009_9.png]
Figure 10
Figure 10. Figure 10: QoR is consistent across LLMs but differ [PITH_FULL_IMAGE:figures/full_fig_p010_10.png]
Figure 11
Figure 11. Figure 11: Even with best LLM (GPT-4), perfor￾mance degrades as Lines of Code for generated out￾put increases [PITH_FULL_IMAGE:figures/full_fig_p011_11.png]
Figure 12
Figure 12. Figure 12: HDLEval-Comb performance degrades for larger Verilog codes across LLMs. It is also interesting to compare across LLMs for a single HDL [PITH_FULL_IMAGE:figures/full_fig_p011_12.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

49 extracted references · 40 canonical work pages

  1. [1]

    https://artificialanalysis.ai/ models/gpt-35-turbo

    Artificial Analysis. https://artificialanalysis.ai/ models/gpt-35-turbo. Online; accessed on April 2024

    work page 2024

  2. [2]

    https:// github.com/MikePopoloski/slang

    slang - SystemVerilog Language Services. https:// github.com/MikePopoloski/slang. Online; accessed on 5 August 2021

    work page 2021

  3. [3]

    website, November 2017

    HDLBits - Verilog Practice. website, November 2017

    work page 2017

  4. [4]

    website, November 2023

    CWhy. website, November 2023

    work page 2023

  5. [5]

    Few-shot training llms for project-specific code-summarization

    Toufique Ahmed and Premkumar Devanbu. Few-shot training llms for project-specific code-summarization. arXiv preprint arXiv:2207.04237 , 2022

    arXiv 2022

  6. [6]

    Chisel: constructing hardware in a scala embedded language

    JonathanBachrach, HuyVo, BrianRichards, YunsupLee, Andrew Waterman, Rimas Avižienis, John Wawrzynek, and Krste Asanović. Chisel: constructing hardware in a scala embedded language. In DAC Design Automation Conference 2012, pages 1212–1221. IEEE, 2012

    work page 2012

  7. [7]

    Teachinglargelanguagemodelstoself-debug, 2023

    XinyunChen, MaxwellLin, NathanaelSchärli, andDenny Zhou. Teachinglargelanguagemodelstoself-debug, 2023

    work page 2023

  8. [8]

    A pythonic approach for rapid hardware prototyping and instrumentation

    John Clow, Georgios Tzimpragos, Deeksha Dangwal, Sammy Guo, Joseph McMahan, and Timothy Sherwood. A pythonic approach for rapid hardware prototyping and instrumentation. In Field Programmable Logic and Ap- plications (FPL), 2017 27th International Conference on , pages 1–7. IEEE, 2017

    work page 2017

Show all 49 references
  1. [9]

    Self- collaboration code generation via chatgpt, 2023

    Yihong Dong, Xue Jiang, Zhi Jin, and Ge Li. Self- collaboration code generation via chatgpt, 2023

    2023

  2. [10]

    Efabless 1st competition winners

    efabless. Efabless 1st competition winners. https:// efabless.com/genai/challenges/1, 2023

    2023

  3. [11]

    Efabless 2nd competition winners

    efabless. Efabless 2nd competition winners. https:// efabless.com/genai/challenges/2-winners, 2023

    2023

  4. [12]

    Improving automatically generated code from codex via automated program repair.arXiv preprint arXiv:2205.10583, 2022

    Zhiyu Fan, Xiang Gao, Abhik Roychoudhury, and Shin Hwei Tan. Improving automatically generated code from codex via automated program repair.arXiv preprint arXiv:2205.10583, 2022

    2022 arXiv

  5. [13]

    XLS Website

    Google. XLS Website. https://github.com/google/xls/, 2022

    2022

  6. [14]

    Financebench: A new benchmark for financial question answering, 2023

    Pranab Islam, Anand Kannappan, Douwe Kiela, Rebecca Qian, Nino Scherrer, and Bertie Vidgen. Financebench: A new benchmark for financial question answering, 2023. 13

    2023

  7. [15]

    Self-planning code generation with large language models, 2023

    Xue Jiang, Yihong Dong, Lecheng Wang, Zheng Fang, Qi- wei Shang, Ge Li, Zhi Jin, and Wenpin Jiao. Self-planning code generation with large language models, 2023

    2023

  8. [16]

    Towards llm-powered verilog rtl assistant: Self- verification and self-correction, 2024

    Amit Kumar, Deepak Singh, Nalini Gupta, and Meena Bhatia. Towards llm-powered verilog rtl assistant: Self- verification and self-correction, 2024

    2024

  9. [17]

    Retrieval-augmented generation for knowledge-intensive nlp tasks, 2021

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen tau Yih, Tim Rocktäschel, Se- bastian Riedel, and Douwe Kiela. Retrieval-augmented generation for knowledge-intensive nlp tasks, 2021

    2021

  10. [18]

    The unlocking spell on base llms: Rethinking alignment via in-context learning, 2023

    Bill Yuchen Lin, Abhilasha Ravichander, Ximing Lu, Nouha Dziri, Melanie Sclar, Khyathi Chandu, Chandra Bhagavatula, and Yejin Choi. The unlocking spell on base llms: Rethinking alignment via in-context learning, 2023

    2023

  11. [19]

    Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning.Advances in Neural In- formation Processing Systems, 35:1950–1965, 2022

    Haokun Liu, Derek Tam, Mohammed Muqeeth, Jay Mo- hta, Tenghao Huang, Mohit Bansal, and Colin A Raf- fel. Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning.Advances in Neural In- formation Processing Systems, 35:1950–1965, 2022

    1950

  12. [20]

    Verilogeval: Evaluating large language models for verilog code generation

    Mingjie Liu, Nathaniel Pinckney, Brucek Khailany, and Haoxing Ren. Verilogeval: Evaluating large language models for verilog code generation. arXiv preprint arXiv:2309.07544, 2023

    2023 arXiv

  13. [21]

    Rtlcoder: Outperforming gpt-3.5 in design rtl generation with our open-source dataset and lightweight solution, 2024

    Shang Liu, Wenji Fang, Yao Lu, Qijun Zhang, Hongce Zhang, and Zhiyao Xie. Rtlcoder: Outperforming gpt-3.5 in design rtl generation with our open-source dataset and lightweight solution, 2024

    2024

  14. [22]

    Rtllm: An open-source benchmark for design rtl generation with large language model, 2023

    Yao Lu, Shang Liu, Qijun Zhang, and Zhiyao Xie. Rtllm: An open-source benchmark for design rtl generation with large language model, 2023

    2023

  15. [23]

    Self-refine: Iterative refinement with self-feedback, 2023

    Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Her- mann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. Self-refine: Iterat...

    2023

  16. [24]

    Coffee: Boost your code llms by fixing bugs with feedback, 2023

    Seungjun Moon, Yongho Song, Hyungjoo Chae, Dongjin Kang, Taeyoon Kwon, Kai Tzu iunn Ong, Seung won Hwang, and Jinyoung Yeo. Coffee: Boost your code llms by fixing bugs with feedback, 2023

    2023

  17. [25]

    Lever: Learning to verify language-to-code generation with exe- cution

    Ansong Ni, Srini Iyer, Dragomir Radev, Ves Stoyanov, Wen-tau Yih, Sida I Wang, and Xi Victoria Lin. Lever: Learning to verify language-to-code generation with exe- cution. arXiv preprint arXiv:2302.08468 , 2023

    2023 arXiv

  18. [26]

    Olausson, Jeevana Priya Inala, Chenglong Wang, Jianfeng Gao, and Armando Solar-Lezama

    Theo X. Olausson, Jeevana Priya Inala, Chenglong Wang, Jianfeng Gao, and Armando Solar-Lezama. Is self-repair a silver bullet for code generation?, 2023

    2023

  19. [27]

    A survey on trans- fer learning

    Sinno Jialin Pan and Qiang Yang. A survey on trans- fer learning. IEEE Transactions on knowledge and data engineering, 22(10):1345–1359, 2010

    2010

  20. [28]

    A study of transfer of skill between programming languages

    Jean Clarice Scholtz. A study of transfer of skill between programming languages. The University of Nebraska- Lincoln, 1989

    1989

  21. [29]

    Desmarais, and Giuliano Antoniol

    Florian Tambon, Arghavan Moradi Dakhel, Amin Nikan- jam, Foutse Khomh, Michel C. Desmarais, and Giuliano Antoniol. Bugs in large language models generated code: An empirical study, 2024

    2024

  22. [30]

    Benchmarking large language models for automated verilog rtl code genera- tion

    Shailja Thakur, Baleegh Ahmad, Zhenxing Fan, Ham- mond Pearce, Benjamin Tan, Ramesh Karri, Brendan Dolan-Gavitt, and Siddharth Garg. Benchmarking large language models for automated verilog rtl code genera- tion. In 2023 Design, Automation and Test in Europe Conference and Exh...

    2023

  23. [31]

    Verigen: A large language model for verilog code generation, 2023

    Shailja Thakur, Baleegh Ahmad, Hammond Pearce, Ben- jamin Tan, Brendan Dolan-Gavitt, Ramesh Karri, and Siddharth Garg. Verigen: A large language model for verilog code generation, 2023

    2023

  24. [32]

    Autochip: Automating hdl generation using llm feedback, 2023

    Shailja Thakur, Jason Blocklove, Hammond Pearce, Ben- jaminTan, SiddharthGarg, andRameshKarri. Autochip: Automating hdl generation using llm feedback, 2023

    2023

  25. [33]

    Rtlfixer: Automatically fixing rtl syntax errors with large language models, 2024

    Yun-Da Tsai, Mingjie Liu, and Haoxing Ren. Rtlfixer: Automatically fixing rtl syntax errors with large language models, 2024

    2024

  26. [34]

    Intervenor: Prompt the coding ability of large language models with the in- teractive chain of repairing, 2023

    Hanbin Wang, Zhenghao Liu, Shuo Wang, Ganqu Cui, Ning Ding, Zhiyuan Liu, and Ge Yu. Intervenor: Prompt the coding ability of large language models with the in- teractive chain of repairing, 2023

    2023

  27. [35]

    Chatcoder: Chat- based refine requirement improves llms’ code generation, 2023

    Zejun Wang, Jia Li, Ge Li, and Zhi Jin. Chatcoder: Chat- based refine requirement improves llms’ code generation, 2023

    2023

  28. [36]

    Chain-of-thought prompting elicits reason- ing in large language models, 2023

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. Chain-of-thought prompting elicits reason- ing in large language models, 2023

    2023

  29. [37]

    Yosys Open SYnthesis Suite

    Clifford Wolf. Yosys Open SYnthesis Suite. https:// github.com/YosysHQ/yosys, 2022. Online; accessed on December 2022

    2022

  30. [38]

    Conver- sational automated program repair

    Chunqiu Steven Xia and Lingming Zhang. Conver- sational automated program repair. arXiv preprint arXiv:2301.13246, 2023

    2023 arXiv

  31. [39]

    Le, Denny Zhou, and Xinyun Chen

    Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V. Le, Denny Zhou, and Xinyun Chen. Large lan- guage models as optimizers, 2023

    2023

  32. [40]

    A new design approach of hardware implemen- tation through natural language entry.IET Collaborative Intelligent Manufacturing, 5(4):e12087, 2023

    Kaiyuan Yang, Haotian Liu, Yuqin Zhao, and Tiantai Deng. A new design approach of hardware implemen- tation through natural language entry.IET Collaborative Intelligent Manufacturing, 5(4):e12087, 2023

    2023

  33. [41]

    React: Synergizing reasoning and acting in language models, 2023

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models, 2023

    2023

  34. [42]

    Hdlde- bugger: Streamlining hdl debugging with large language models, 2024

    Xufeng Yao, Haoyang Li, Tsz Ho Chan, Wenyi Xiao, Mingxuan Yuan, YuHuang, Lei Chen, andBei Yu. Hdlde- bugger: Streamlining hdl debugging with large language models, 2024

    2024

  35. [43]

    Lumos: Learning agents with unified data, modular design, and open-source llms, 2023

    DaYin, FaezeBrahman, AbhilashaRavichander, Khyathi Chandu, Kai-Wei Chang, Yejin Choi, and Bill Yuchen Lin. Lumos: Learning agents with unified data, modular design, and open-source llms, 2023

    2023

  36. [44]

    HDLEval Benchmarking LLMs for Multiple HDLs

    MarkZakharov, FarzanehRabieiKashanaki, andJoseRe- nau. HDLEval Benchmarking LLMs for Multiple HDLs. In 1st IEEE International Workshop on LLM-Aided De- sign, 2024

    2024

  37. [45]

    Self- edit: Fault-awarecodeeditorforcodegeneration

    Kechi Zhang, Zhuo Li, Jia Li, Ge Li, and Zhi Jin. Self- edit: Fault-awarecodeeditorforcodegeneration. InAnna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki, edi- tors, Proceedings of the 61st Annual Meeting of the As- sociation for Computational Linguistics (Volume 1: Long Pap...

    2023

  38. [46]

    Expel: Llm agents are experiential learners

    Andrew Zhao, Daniel Huang, Quentin Xu, Matthieu Lin, Yong-Jin Liu, and Gao Huang. Expel: Llm agents are experiential learners. arXiv preprint arXiv:2308.10144 , 2023

    2023 arXiv

  39. [47]

    Verilogcoder: Autonomous verilog coding agents with graph-based planning and abstract syntax tree (ast)-based waveform tracing tool, 2024

    Jun Zhao, Min Lee, Ananya Rastogi, Huan Yu, Samuel Chen, and Li Xiang. Verilogcoder: Autonomous verilog coding agents with graph-based planning and abstract syntax tree (ast)-based waveform tracing tool, 2024

    2024

  40. [48]

    Can chatgpt replace stack- overflow? a study on robustness and reliability of large language model code generation, 2024

    Li Zhong and Zilong Wang. Can chatgpt replace stack- overflow? a study on robustness and reliability of large language model code generation, 2024

    2024

  41. [49]

    Agents: An open-source frame- work for autonomous language agents, 2023

    Wangchunshu Zhou, Yuchen Eleanor Jiang, Long Li, Jia- long Wu, Tiannan Wang, Shi Qiu, Jintian Zhang, Jing Chen, Ruipu Wu, Shuai Wang, Shiding Zhu, Jiyu Chen, Wentao Zhang, Ningyu Zhang, Huajun Chen, Peng Cui, and Mrinmaya Sachan. Agents: An open-source frame- work for autonomo...

    2023

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.