Pith. sign in

REVIEW 28 cited by

AutoChip: Automating HDL Generation Using LLM Feedback

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2311.04887 v2 pith:KYUJXCPM submitted 2023-11-08 cs.PL

Shailja Thakur , Jason Blocklove , Hammond Pearce , Benjamin Tan , Siddharth Garg , Ramesh Karri This is my paper · ORCID
classification cs.PL
keywords verilogautochipllmscodeaccurategenerateautomatingcontext
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Traditionally, designs are written in Verilog hardware description language (HDL) and debugged by hardware engineers. While this approach is effective, it is time-consuming and error-prone for complex designs. Large language models (LLMs) are promising in automating HDL code generation. LLMs are trained on massive datasets of text and code, and they can learn to generate code that compiles and is functionally accurate. We aim to evaluate the ability of LLMs to generate functionally correct HDL models. We build AutoChip by combining the interactive capabilities of LLMs and the output from Verilog simulations to generate Verilog modules. We start with a design prompt for a module and the context from compilation errors and debugging messages, which highlight differences between the expected and actual outputs. This ensures that accurate Verilog code can be generated without human intervention. We evaluate AutoChip using problem sets from HDLBits. We conduct a comprehensive analysis of the AutoChip using several LLMs and problem categories. The results show that incorporating context from compiler tools, such as Icarus Verilog, improves the effectiveness, yielding 24.20% more accurate Verilog. We release our evaluation scripts and datasets as open-source contributions at the following link https://github.com/shailja-thakur/AutoChip.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 28 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. ArchEval: Measuring AI Agents as Computer Architects

    cs.AR 2026-07 conditional novelty 7.0 of 10

    LLM agents beat architecture baselines with full simulator harnesses, but only one configuration stays above baseline without feedback, and performance modeling remains weak.

  2. Interpretable and Verifiable Hardware Generation with LLM-Driven Stepwise Refinement

    cs.SE 2026-06 unverdicted novelty 7.0 of 10

    Framework uses LLM-driven stepwise application of transformation rules to generate verifiable RTL hardware designs from specifications.

  3. LACE: Large Language Model Aided Multi-Agent Framework for Agile RISC-V Instruction Extension

    cs.AR 2026-08 conditional novelty 6.0 of 10

    LACE uses multiple LLM agents to translate RISC-V instruction intentions into hardware modifications, raising end-to-end generation success from near zero to 72.8 percent on four embedded cores.

  4. VClare: Resolving Imperfect Specifications in LLM-Based Verilog Generation

    cs.AR 2026-07 conditional novelty 6.0 of 10

    Repairing imperfect Verilog specs with LLM inconsistency mining and simulation clustering lifts generation pass rates ~13%, with sim-level repair remaining robust as designs grow complex.

  5. DRC-Aid: Design-Rule Correction via Agentic Framework utilizing Inference-Time Large Language Models

    cs.AR 2026-07 conditional novelty 6.0 of 10

    A closed-loop agentic framework using LLM action selection over a rule-engine menu repairs ~92.5% of DRC-violating FreePDK45 layouts to DRC-clean, LVS-equivalent states.

  6. MACO: A Multi-Agent LLM Framework for Automated CGRA Hardware/Software Co-Design

    cs.AR 2025-09 reject novelty 6.0 of 10

    A multi-agent LLM framework that iteratively co-designs CGRA hardware and software parameters, reporting power and performance improvements over LLM and manual baselines.

  7. AutoVeriFix: Automatically Correcting Errors and Enhancing Functional Correctness in LLM-Generated Verilog Code

    cs.AR 2025-09 conditional novelty 6.0 of 10

    AutoVeriFix improves LLM-generated Verilog functional correctness by generating a high-level Python reference model, deriving a high-coverage testbench, and iteratively fixing Verilog simulation mismatches.

  8. FedChip: Federated LLM for Artificial Intelligence Accelerator Chip Design

    cs.AR 2025-07 conditional novelty 6.0 of 10

    FedChip applies federated fine-tuning to LLM-based AI accelerator design, adding a 30k-sample dataset and a Chip@k metric, with a reported 77% quality improvement over high-end LLMs.

  9. MMCircuitEval: A Comprehensive Multimodal Circuit-Focused Benchmark for Evaluating LLMs

    cs.LG 2025-07 conditional novelty 6.0 of 10

    A new multimodal benchmark of 3,614 circuit QA pairs shows that large language models perform worst on back-end layout and computation tasks, and that current models generally underperform on circuit design questions.

  10. Free and Fair Hardware: A Pathway to Copyright Infringement-Free Verilog Generation using LLMs

    cs.AI 2025-05 conditional novelty 6.0 of 10

    A new Verilog dataset with license and copyright filtering, a similarity-based copyright violation benchmark, and a fine-tuned Llama model (FreeV) with a 3% reported violation rate and VerilogEval pass@10 gains.

  11. Veritas: Deterministic Verilog Code Synthesis from LLM-Generated Conjunctive Normal Form

    cs.AR 2025-05 conditional novelty 6.0 of 10

    Veritas fine-tunes Llama-3.2-3B to emit CNF formulas for small circuits and converts them to Verilog, reporting perfect accuracy on a small held-out set without releasing artifacts.

  12. SysVCoder: An LLM-Driven Framework for Systematic Generation of System-Level Design

    cs.SE 2025-04 conditional novelty 6.0 of 10

    A two-stage LLM pipeline converts natural-language hardware specs into a structured intermediate representation and then into system-level Verilog, beating open-source Verilog generators and matching a GPT-4 agent wit...

  13. VRank: Enhancing Verilog Code Generation from Large Language Models via Self-Consistency

    cs.AR 2025-01 conditional novelty 6.0 of 10

    VRank selects better LLM-generated Verilog code by clustering candidates that produce identical simulation outputs and ranking the clusters by consistency, yielding an average 10.5% pass@1 gain on VerilogEval-Human.

  14. Paradigm-Based Automatic HDL Code Generation Using LLMs

    cs.PL 2025-01 conditional novelty 6.0 of 10

    A paradigm-based workflow with information-list reuse and a two-phase loop improves LLM-generated Verilog pass rates on VerilogEval, with the full-dataset result built from a hybrid of baseline and proposed-method outputs.

  15. C2HLSC: Leveraging Large Language Models to Bridge the Software-to-Hardware Design Gap

    cs.AR 2024-11 conditional novelty 6.0 of 10

    An LLM-based, feedback-driven pipeline rewrites generic C programs into HLS-synthesizable C, succeeding on most but not all of ten real-world benchmarks.

  16. RTL-Breaker: Assessing the Security of LLMs against Backdoor Attacks on HDL Code Generation

    cs.CR 2024-11 conditional novelty 6.0 of 10

    RTL-Breaker shows that fine-tuning an LLM on a small fraction of poisoned Verilog samples can make it generate malicious RTL code when a specific trigger appears, while still passing VerilogEval functional checks.

  17. Masala-CHAI: A Large-Scale SPICE Netlist Dataset for Analog Circuits by Harnessing AI

    cs.AR 2024-11 conditional novelty 6.0 of 10

    Masala-CHAI automatically converts schematic images into SPICE netlists using object detection, line detection, and LLMs, producing a 7,500-example open dataset that boosts analog netlist generation Pass@1 by up to 46...

  18. CorrectBench: Automatic Testbench Generation with Functional Self-Correction using LLMs for HDL Design

    cs.SE 2024-11 conditional novelty 6.0 of 10

    A self-validating and self-correcting LLM framework raises testbench generation pass rate to 70.13%, up from 52.18% for the prior AutoBench framework.

  19. ChipVerilog: A Large-Scale OpenCores-Derived Benchmark for LLM-Based Verilog RTL Generation

    cs.AR 2026-07 conditional novelty 5.0 of 10

    On 64 large OpenCores-derived Verilog tasks, top LLMs reach 23.6% functional pass@1, 37.5% pass@5, and 0% on designs with two or more submodules, showing hierarchical RTL generation remains unsolved.

  20. EvoVerilog: Large Langugage Model Assisted Evolution of Verilog Code

    cs.AR 2025-06 unverdicted novelty 5.0 of 10

    EvoVerilog uses multiobjective evolutionary search with LLMs to generate Verilog code, reporting higher pass@10 than prior methods on VerilogEval-Machine and VerilogEval-Human.

  21. AiEDA: Agentic AI Design Framework for Digital ASIC System Design

    cs.AR 2024-12 reject novelty 5.0 of 10

    AiEDA combines LLM agents with open-source EDA tools in a four-stage concept-to-GDSII flow, but it has not yet demonstrated the full flow on its KWS case study.

  22. Unleashing GHOST: An LLM-Powered Framework for Automated Hardware Trojan Design

    cs.CR 2024-12 conditional novelty 5.0 of 10

    An LLM-powered framework can automatically insert functional, synthesizable hardware Trojans into RTL chip designs, and a state-of-the-art ML detector fails to catch them.

  23. VerilogDB: The Largest, Highest-Quality Dataset with a Preprocessing Framework for LLM-based RTL Generation

    cs.AR 2025-07 conditional novelty 4.0 of 10

    A new pipeline and dataset of 20,392 synthesis-checked Verilog modules for LLM fine-tuning is presented, claimed to be the largest high-quality dataset of its kind.

  24. QiMeng: Fully Automated Hardware and Software Design for Processor Chip

    cs.AR 2025-06 conditional novelty 4.0 of 10

    QiMeng is a proposed three-layer architecture for automating processor hardware and software design, with several published components but no integrated implementation yet.

  25. Customizing a Large Language Model for VHDL Design of High-Performance Microprocessors

    cs.AR 2025-05 conditional novelty 4.0 of 10

    Extended pretraining on proprietary VHDL code and documents improved a Granite LLM's expert-rated explanations of microprocessor VHDL from 43% to 69%, and instruction tuning pushed the expected rating to 71%.

  26. A Survey of Research in Large Language Models for Electronic Design Automation

    cs.LG 2025-01 conditional novelty 2.0 of 10

    A survey of LLM applications in electronic design automation, organized by design stage and adaptation technique.

  27. Revolution or Hype? Seeking the Limits of Large Models in Hardware Design

    cs.LG 2025-09 conditional novelty 1.0 of 10

    Large models can help early-stage hardware design and verification, but their reliability, data, and precision limits mean traditional EDA algorithms and formal verification remain necessary.

  28. Tools and Methodologies for System-Level Design

    cs.AR 2025-07 unverdicted

    A pedagogical review of system-level design tools and methodologies for heterogeneous systems-on-chips.

Pith tools