ProtocolLLM: RTL Benchmark for SystemVerilog Generation of Communication Protocols

Arnav Sheth; Ivaxi Sheth; Mario Fritz

REVIEW 3 major objections 5 minor 1 cited by

ProtocolLLM: RTL Benchmark for SystemVerilog Generation of Communication Protocols

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Current LLMs cannot reliably produce timing-correct SystemVerilog for communication protocols; even the best models pass waveform checks on only a minority of trials.

desk verdict Useful benchmark, plausible headline, but the unvalidated scoreboards and tiny task counts make exact pass rates provisional until the evaluation artifacts ship. read the letter →

arxiv 2506.07945 v2 pith:K3A2GF6E submitted 2025-06-09 cs.AR cs.AIcs.CL

Arnav Sheth , Ivaxi Sheth , Mario Fritz This is my paper

classification cs.ARcs.AIcs.CL

keywords LLMSystemVerilogRTLgenerationcommunicationprotocolshardwareverificationwaveformanalysistimingconstraintsbenchmark

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to test whether large language models can generate synthesizable SystemVerilog code for communication protocols that respect real timing constraints, not just syntax. It introduces ProtocolLLM, a benchmark covering SPI, I2C, UART, and AXI with a three-stage evaluation: linting, logic synthesis, and waveform-level simulation against UVM testbenches. Across ten open-source and proprietary models, the benchmark finds that most generated designs fail waveform checks, and the best models—GPT-4.1 and Gemini 2.5 Pro—still pass timing-accurate checks on only a minority of protocol trials. The paper argues that syntax and synthesis checks alone overstate LLM hardware capability, and that timing-correct protocol generation is the real bottleneck for automated RTL design.

What carries the argument

The central object is the benchmark itself plus its waveform-validation stage. ProtocolLLM combines four real protocols (SPI with all four CPOL/CPHA modes, I2C with 7- and 10-bit addressing, UART with and without parity, and AXI4-Lite) with two prompt settings (standard and specification-retrieval-augmented). The load-bearing mechanism is a three-stage funnel: a lint check for syntax, logic synthesis for synthesizability and power-area metrics, and UVM-based simulation with scoreboards that compares generated waveform behavior against golden protocol semantics. Only code that survives all three stages counts as a pass, and Pass@1 with $k=1$ mirrors a single-shot, one-attempt usage of a code assistant.

What would settle it

Run a known-correct, hand-verified SystemVerilog implementation of one protocol (for example, an I2C master from a commercial IP library) through the ProtocolLLM pipeline. If that implementation fails the waveform stage, the testbench is not a faithful oracle, and the models' low pass rates could be artifacts of an over-strict or buggy golden reference.

Watch

Extended reading notes

Core claim

ProtocolLLM's central claim is that current LLMs cannot reliably generate SystemVerilog RTL for communication protocols that meet waveform-level timing and handshake semantics. The benchmark's waveform stage, built on UVM testbenches with scoreboards, exposes failures that linting and synthesis miss, including inferred latches, incomplete state transitions, handshake signals left asserted, and timing violations. On the headline Pass@1 metric, most models score zero on waveform correctness for most protocols; even the strongest model, GPT-4.1, passes the waveform stage on only two of the four protocols, with SPI at 0.75 and UART at 1.0 in the no-spec setting, while Gemini 2.5 Pro succeeds on SPI and some AXI variants. The paper thus establishes not only that LLMs struggle, but that the difficulty is concentrated in temporal behavior rather than in producing syntactically valid HDL.

Load-bearing premise

The results stand on the assumption that the benchmark's UVM testbenches and scoreboards faithfully capture the full timing and handshake semantics of each protocol. If those golden references contain bugs or miss corner cases, the waveform Pass@1 numbers stop measuring true protocol correctness.

Editorial extensions

If this is right

HDL benchmarks that check only syntax and synthesis overstate LLM competence; timing-aware verification is necessary before generated RTL can be trusted.
Chip teams should treat LLM-generated protocol RTL as a draft requiring human or formal repair, not as deployable IP.
Retrieval-augmented prompting with datasheets can improve waveform correctness even when it slightly lowers lint and synthesis pass rates.
Code-specialized models are not automatically good at HDL; some emitted Python instead of SystemVerilog, indicating a domain mismatch that general-purpose frontier models handle better.
Protocol complexity matters for evaluation: SPI and AXI were generated more reliably than I2C and UART, so benchmarks should stratify tasks by timing complexity.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The same waveform-first evaluation could be applied to other timing-critical blocks such as bus arbiters, memory controllers, and interface bridges, potentially revealing similar failure patterns.
The near-miss failures described (handshake signals stuck high, missing initial reads) suggest that an automated repair pass—lint-fixing, formal verification, or constraint-guided patching—might recover a meaningful share of currently failing outputs.
The ordering of protocol difficulty (SPI and AXI above I2C and UART) may reflect the abundance of each protocol in LLM training corpora; if so, a small amount of hardware-specific fine-tuning could shift the ranking, a testable prediction implied by the paper's call for domain-aware models.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Here's the short version. ProtocolLLM fills a real gap: it's the first benchmark that asks LLMs to generate SystemVerilog for whole communication protocols (SPI, I2C, UART, AXI-Lite) and checks the result with waveform-level simulation, not just syntax or synthesis. That is genuinely useful for the ML-for-HW crowd, and the three-stage pipeline is a sensible way to separate structural from temporal failures. The headline result, that most models fail to produce timing-correct protocol RTL, is also plausible: in Table II, almost every waveform Pass@1 is zero, and GPT-4.1 on SPI (0.75) is the only real success.

The problem is that the evaluation's load-bearing part, the UVM scoreboards, is not described in enough detail to trust the exact numbers. We don't see the testbench source, no coverage or assertion counts are reported, and there's no independent validation of the scoreboards against a known-good implementation. A scoreboard that is too strict (expects a specific FSM structure or exact cycle alignment) would falsely fail functionally correct code; one that only samples data values at the right times would miss handshake or timing violations. Both would change the headline. The per-protocol task counts are also tiny (a handful of variants per protocol), so the per-model comparisons are fragile. The paper does link to a GitHub repo, so these artifacts may be out there; but they need to be in the paper (or openly released and described) before the pass rates can be taken at face value.

The writing is also rougher than it should be, with typos like 'wholistically' and 'constrains' throughout, but that's minor compared to the evaluation question.

Bottom line: this paper deserves a serious referee. The benchmark idea is good, the results are interesting and likely directionally correct, and the main concern is fixable in revision. The authors should be asked to release the testbenches, report coverage, and validate the scoreboards against a reference design or formal properties, and to either add more variants or soften per-protocol claims. I wouldn't cite it in my own work, but I'd send it to someone who works on RTL generation benchmarks.

Referee Report

3 major / 5 minor

Summary. The paper introduces ProtocolLLM, a benchmark suite that asks LLMs to generate complete, synthesizable SystemVerilog modules for the SPI, I²C, UART, and AXI-Lite communication protocols, given either a standard prompt or a prompt augmented with retrieved specification text. Generated code is evaluated in three stages: Verilator linting, Yosys logic synthesis, and UVM-based waveform simulation against protocol-specific scoreboards. Pass@1 rates are reported for ten open-source and proprietary models. The central empirical finding is that while many models pass syntax and synthesis checks for some protocols, most models fail the waveform checks on most protocol tasks, leading the authors to conclude that current LLMs cannot reliably produce timing-correct protocol RTL.

Significance. If the benchmark artifacts are released and the golden testbenches are validated, ProtocolLLM fills a genuine gap by extending LLM-HDL evaluation beyond syntax and synthesis to protocol-level timing behavior. The three-stage pipeline of linting, synthesis, and waveform verification is a sensible and more rigorous evaluation design than syntax-only benchmarks, and the use of temperature 0, open-source EDA tools, and a comparison of code-specialized versus general-purpose models are strengths. The qualitative observation that syntax and synthesis success does not imply waveform success is valuable and likely robust. However, the lack of testbench validation and the very small number of tasks per protocol currently limit the reliability of the quantitative pass rates and the inter-model comparisons, so the exact ranking and the precise failure rates should be treated with caution.

major comments (3)

[Section III-B and Table II] The headline claim that most models fail timing-accurate waveform checks rests entirely on the UVM scoreboards described only in Section III-B, but the manuscript does not provide the testbench source, report any functional coverage or assertion counts, or validate the scoreboards against a known-good reference implementation or against explicit timing requirements from the protocol datasheets. Without this calibration, the waveform Pass@1 rates are ambiguous: a scoreboard that is too strict (for example, expecting a particular FSM structure or exact-cycle output alignment) would falsely fail functionally correct designs, while a too-permissive scoreboard could pass designs that violate handshake or setup/hold behavior. Both errors would directly change the pass rates in Table II and the central conclusion, so the authors should release the testbenches and demonstrate their fidelity, for example by showing that hand-written reference implementations pass and that mutations introducing known timing bugs fail.
[Section III-A and Table II] The number of task instances per protocol is very small: SPI has four CPOL/CPHA variants, I²C has two addressing variants, UART has two parity variants, and AXI-Lite appears to be a single task. Since Pass@1 is averaged over these few variants, the reported rates are extremely coarse (multiples of 0.25, 0.5, or 1.0), and no confidence intervals or per-variant results are given. Consequently, the quantitative comparisons in Section IV, including the statements that GPT-4.1 'has the highest pass rate' and that 'bigger models ... tend to outperform all smaller models across all evaluation axes,' are not statistically established; a single variant flip would change several rankings. The authors should report per-variant results and the number of trials per cell, or explicitly frame the results as preliminary.
[Section IV (Overall Results)] The claim that 'most of the models fail to generate SystemVerilog code for communication protocols that follow timing constraints' is broadly supported by the many zero waveform entries in Table II, but the paper overstates some supporting observations. In particular, the text says RAG 'often led to improved functional correctness,' yet Table II shows mixed and often negative effects: GPT-4.1 UART drops from 1.0 without spec to 0 with spec, GPT-o3 AXI drops from 1.0 to 0, and most SPI waveform rates are unchanged or lower with spec. The authors should either resolve this inconsistency with per-task data or soften the claim, because as written the narrative is not consistent with the reported numbers.

minor comments (5)

[Section III-B (Spec-Assisted Generation)] The retrieval-augmented generation setup is not specified: the manuscript does not state which retriever, chunking strategy, or document selection procedure is used, so the 'with spec' condition is not reproducible.
[Throughout] The manuscript contains numerous typographical errors, including 'timing constrains' in the abstract and conclusion, 'ProcotcolLLM' in Section III, 'comparision' and 'finetunened' in Section V, and 'pupose' in Section IV; these should be corrected.
[Figure 2] The sample prompt in Figure 2 contains 'Input logic SPI_start' with inconsistent capitalization and 'parametersas' with missing spacing, and the figure caption does not clearly separate the question text from the module definition; the prompt format should be cleaned up.
[Table II] The power and area columns report values only for designs that reached synthesis, but the table does not indicate whether a 0 entry means the design failed an earlier stage or that no data was collected; adding a marker or footnote would improve interpretability.
[Section V (Conclusion)] The novelty claim that this is 'the first benchmark specifically for SystemVerilog code and the first to focus on communication protocol implementations' should be reconciled with related work, particularly Englhardt et al. in Table I, which covers I²C in an embedded-system context; a more precise comparison would avoid overclaiming.

Circularity Check

0 steps flagged · score 0.0 of 10

Benchmark correctness criteria are externally grounded in protocol datasheets; no circular steps identified.

full rationale

ProtocolLLM is an empirical benchmark paper rather than a derivation chain, and its load-bearing claim (that most models fail to generate SystemVerilog that respects timing constraints) is supported by a three-stage evaluation whose ground truth is external to the models being tested. The waveform stage, which carries the headline conclusion, uses 'UVM-based testbenches' whose scoreboards 'compare expected and actual behavior' where the expected behavior is defined by 'the protocol specification' and protocol timing diagrams (Section III-B), citing the SPI [27], I2C [36], UART [19], and AXI [3] standards. The pass/fail criteria are therefore not defined in terms of the generated outputs, no parameter is fitted to any subset of the model generations, and the Pass@1 metric is a standard deployment-faithful measurement adopted from prior work [11], not a quantity forced by construction. There are no self-citations: none of the 38 references is authored by Sheth, Sheth, or Fritz, so no load-bearing premise is imported from the authors' own prior claims, and no uniqueness or ansatz result is invoked from their own work. The RAG condition, which supplies specification documents to the prompt, is also checked against the same externally specified testbenches, which is a fair task definition rather than circularity. The legitimate concerns raised by a skeptical reading, namely that the author-written scoreboards are not validated against independent reference IP, that no coverage or assertion counts are reported, and that per-protocol variant counts are small, are threats to the validity and robustness of the reported pass rates; they do not constitute circularity under the standard that requires an exhibited reduction of the result to its inputs. The empirical claim could be wrong if the scoreboards mis-encode the protocols, but it is not true by construction, so the honest finding is no significant circularity.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The benchmark's conclusions rest on testbench correctness and toolchain representativeness, not on any fitted parameters or new entities.

assumptions (3)

domain assumption Verilator lint, Yosys synthesis, and UVM waveform simulation are sufficient to judge whether generated RTL is synthesizable and protocol-correct.
The entire evaluation pipeline rests on this assumption. If any gate is too lenient or too strict, the Pass@1 rates misrepresent model capabilities.
domain assumption The hand-written UVM testbenches encode the SPI, I2C, UART, and AXI specifications completely and correctly.
The paper provides no independent validation of the testbenches against reference IP, formal assertions, or hand-written golden RTL.
domain assumption The selected protocols, module interfaces, and prompt styles are representative of real communication-protocol design tasks.
The benchmark's utility depends on the chosen tasks generalizing to industrial RTL work, which the paper asserts but does not demonstrate.

how reviews work

0 comments

Cite this review

Pith. "Pith review of ProtocolLLM: RTL Benchmark for SystemVerilog Generation of Communication Protocols." pith.science (2026). https://pith.science/paper/K3A2GF6E

@misc{pith2026250607945,
  author       = {Pith},
  title        = {Pith review of: ProtocolLLM: RTL Benchmark for SystemVerilog Generation of Communication Protocols},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/K3A2GF6E}},
  note         = {Machine review of arXiv:2506.07945}
}

read the original abstract

Recent advances in large language models (LLMs) have demonstrated strong performance in generating code for general-purpose programming languages. However, their potential for hardware description languages (HDLs), such as SystemVerilog, remains largely unexplored. HDL code generation poses unique challenges due to strict timing semantics, concurrency, and synthesizability constraints essential for correct hardware functionality. Further, HDL-based design flows encompass a broad set of tasks beyond structural code generation, including testbench development, assertion-based verification, timing closure, and protocol-level integration for on-chip communication. In this work, we evaluate the capabilities of both open-source and state-of-the-art LLMs in generating synthesizable and functionally accurate SystemVerilog implementations of widely used communication protocols that are critical components of embedded and System-on-Chip (SoC) systems. We introduce ProtocolLLM, the first benchmark suite specifically targeting these protocols with tasks spanning multiple design abstraction levels and varying prompt specificity. Our evaluation method also focuses on timing correctness in addition to synthesizability and syntactic correctness. We observe that most of the models fail to generate SystemVerilog code for communication protocols that follow timing constrains.

Figures

Figures reproduced from arXiv: 2506.07945 by the authors.

**Figure 2.** Sample prompt for SPI protocol. • Standard Prompting: The model generates HDL code based on a full problem description and module specification (see [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

LLM-Empowered Agentic MAC Protocols: A Dynamic Stackelberg Game Approach
cs.AI 2025-10 conditional novelty 5.0 of 10

An LLM-based multi-agent reinforcement learning framework that models uplink MAC scheduling as a Stackelberg game reports 77.6% higher throughput and 65.2% better fairness in simulation.

Reference graph

Works this paper leans on

39 extracted references · 18 canonical work pages · cited by 1 Pith paper

[1]

Claude opus

“Claude opus.” [Online]. Available: https://www.anthropic.com/claude/ opus

work page
[2]

Claude sonnet

“Claude sonnet.” [Online]. Available: https://www.anthropic.com/claude/ sonnet

work page
[3]

ARM, Introduction to AMBA AXI4 , ARM limited, 2020

work page 2020
[4]

Pyhdl- eval: An llm evaluation framework for hardware design using python- embedded dsls,

C. Batten, N. Pinckney, M. Liu, H. Ren, and B. Khailany, “Pyhdl- eval: An llm evaluation framework for hardware design using python- embedded dsls,” in 2024 ACM/IEEE 6th Symposium on Machine Learn- ing for CAD (MLCAD) , 2024, pp. 1–17

work page 2024
[5]

Smart communication in a wired sensor- and actuator-network of a modular robot actuator system using a hop- protocol with delta-routing,

S. Bosse and D. Lehmhus, “Smart communication in a wired sensor- and actuator-network of a modular robot actuator system using a hop- protocol with delta-routing,” in Proceedings of smart systems integration conference, Como, Italy , 2010, pp. 23–24

work page 2010
[6]

Chipgpt: How far are we from natural language hardware design,

K. Chang, Y . Wang, H. Ren, M. Wang, S. Liang, Y . Han, H. Li, and X. Li, “Chipgpt: How far are we from natural language hardware design,” arXiv preprint arXiv:2305.14019 , 2023

arXiv 2023
[7]

Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities,

G. Comanici, E. Bieber, M. Schaekermann et al., “Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities,” 2025. [Online]. Available: https://arxiv.org/abs/2507.06261

arXiv 2025
[8]

Exploring and characterizing large language models for embedded system development and debugging,

Z. Englhardt, R. Li, D. Nissanka, Z. Zhang, G. Narayanswamy, J. Breda, X. Liu, S. Patel, and V . Iyer, “Exploring and characterizing large language models for embedded system development and debugging,”

work page

Show all 39 references

[9]

Assertllm: Generating and evaluating hardware verification assertions from design specifications via multi-llms,

W. Fang, M. Li, M. Li, Z. Yan, S. Liu, Z. Xie, and H. Zhang, “Assertllm: Generating and evaluating hardware verification assertions from design specifications via multi-llms,” 2024. [Online]. Available: https://arxiv.org/abs/2402.00386

2024
[10]

All artificial, less intelligence: Genai through the lens of formal verification,

D. N. Gadde, A. Kumar, T. Nalapat, E. Rezunov, and F. Cappellini, “All artificial, less intelligence: Genai through the lens of formal verification,” arXiv preprint arXiv:2403.16750 , 2024

2024 arXiv
[11]

Tur- tle: A unified evaluation of llms for rtl generation,

D. Garcia-Gasulla, G. Kestor, E. Parisi, M. Albert ´ı-Binimelis, C. Gutier- rez, R. M. Ghorab, O. Montenegro, B. Homs, and M. Moreto, “Tur- tle: A unified evaluation of llms for rtl generation,” arXiv preprint arXiv:2504.01986, 2025

2025 arXiv
[12]

Towards llm-powered verilog rtl assistant: Self-verification and self-correction,

H. Huang, Z. Lin, Z. Wang, X. Chen, K. Ding, and J. Zhao, “Towards llm-powered verilog rtl assistant: Self-verification and self-correction,” arXiv preprint arXiv:2406.00115 , 2024

2024 arXiv
[13]

Opencoder: The open cookbook for top-tier code large language models,

S. Huang, T. Cheng, J. K. Liu, J. Hao, L. Song, Y . Xu, J. Yang, J. Liu, C. Zhang, L. Chai et al., “Opencoder: The open cookbook for top-tier code large language models,” arXiv preprint arXiv:2411.04905 , 2024

2024 arXiv
[14]

Qwen2. 5-coder technical report,

B. Hui, J. Yang, Z. Cui, J. Yang, D. Liu, L. Zhang, T. Liu, J. Zhang, B. Yu, K. Lu et al. , “Qwen2. 5-coder technical report,” arXiv preprint arXiv:2409.12186, 2024

2024 arXiv
[15]

Automated synthesis of hardware designs using symbolic feedback and grammar-constrained decoding in large language models,

S. K. Jha, S. Jha, M. R. H. Rashed, R. Ewetz, and A. Velasquez, “Automated synthesis of hardware designs using symbolic feedback and grammar-constrained decoding in large language models,” in NAECON 2024-IEEE National Aerospace and Electronics Conference . IEEE, 2024, pp. 95–100

2024
[16]

A survey on large language models for code generation,

J. Jiang, F. Wang, J. Shen, S. Kim, and S. Kim, “A survey on large language models for code generation,” arXiv preprint arXiv:2406.00515, 2024

2024 arXiv
[17]

Swe-bench: Can language models resolve real-world github issues?

C. E. Jimenez, J. Yang, A. Wettig, S. Yao, K. Pei, O. Press, and K. Narasimhan, “Swe-bench: Can language models resolve real-world github issues?” ICLR, 2024

2024
[18]

Hdleval benchmarking llms for multiple hdls,

F. R. Kashanaki, M. Zakharov, and J. Renau, “Hdleval benchmarking llms for multiple hdls,” in 2024 IEEE LLM Aided Design Workshop (LAD), 2024, pp. 1–5

2024
[19]

Uart communication protocol and how it works,

Kelvin, “Uart communication protocol and how it works,” https://www.seeedstudio.com/blog/2022/09/08/uart-communication- protocol-and-how-it-works/, 2022, accessed: 16th April 2025

2022
[20]

Interfacing of mems motion sensor with fpga using i2c protocol,

R. S. S. Kumari and C. Gayathri, “Interfacing of mems motion sensor with fpga using i2c protocol,” in 2017 International Conference on Innovations in Information, Embedded and Communication Systems (ICIIECS). IEEE, 2017, pp. 1–5

2017
[21]

Autosilicon: Scaling up rtl design generation capability of large language models,

C. Li, C. Chen, Y . Pan, W. Xu, Y . Liu, K. Chang, Y . Wang, M. Wang, Y . Wang, H. Li et al. , “Autosilicon: Scaling up rtl design generation capability of large language models,” ACM Transactions on Design Automation of Electronic Systems , 2025

2025
[22]

Starcoder: may the source be with you!

R. Li, L. B. Allal, Y . Zi, N. Muennighoff, D. Kocetkov, C. Mou, M. Marone, C. Akiki, J. Li, J. Chim et al., “Starcoder: may the source be with you!” arXiv preprint arXiv:2305.06161 , 2023

2023 arXiv
[23]

Verilogeval: Evaluating large language models for verilog code generation,

M. Liu, N. Pinckney, B. Khailany, and H. Ren, “Verilogeval: Evaluating large language models for verilog code generation,” in 2023 IEEE/ACM International Conference on Computer Aided Design (ICCAD) . IEEE, 2023, pp. 1–8

2023
[24]

Starcoder 2 and the stack v2: The next generation,

A. Lozhkov, R. Li, L. B. Allal, F. Cassano, J. Lamy-Poirier, N. Tazi, A. Tang, D. Pykhtar, J. Liu, Y . Wei et al. , “Starcoder 2 and the stack v2: The next generation,” arXiv preprint arXiv:2402.19173 , 2024

2024 arXiv
[25]

Rtllm: An open-source benchmark for design rtl generation with large language model,

Y . Lu, S. Liu, Q. Zhang, and Z. Xie, “Rtllm: An open-source benchmark for design rtl generation with large language model,” 2023. [Online]. Available: https://arxiv.org/abs/2308.05345

2023 arXiv
[26]

Rtllm: An open-source benchmark for design rtl generation with large language model,

——, “Rtllm: An open-source benchmark for design rtl generation with large language model,” in 2024 29th Asia and South Pacific Design Automation Conference (ASP-DAC). IEEE, 2024, pp. 722–727

2024
[27]

Serial peripheral interface (spi),

MikeGrusin, “Serial peripheral interface (spi),” https://learn.sparkfun. com/tutorials/serial-peripheral-interface-spi/all, accessed: 16th April 2025

2025
[28]

Gpt-4.1,

OpenAI, “Gpt-4.1,” 2025

2025
[29]

Gpt-4 technical report,

e. a. OpenAI, “Gpt-4 technical report,” 2024. [Online]. Available: https://arxiv.org/abs/2303.08774

2024 arXiv
[30]

A survey of research in large language models for electronic design automation,

J. Pan, G. Zhou, C.-C. Chang, I. Jacobson, J. Hu, and Y . Chen, “A survey of research in large language models for electronic design automation,” ACM Transactions on Design Automation of Electronic Systems , 2025

2025
[31]

Usart, spi, i2c, and communication protocols,

A. Subero, “Usart, spi, i2c, and communication protocols,” in Program- ming PIC Microcontrollers with XC8: Mastering Classical Embedded Design. Springer, 2024, pp. 297–366

2024
[32]

Evaluating large language models for automatic register transfer logic generation via high-level synthesis,

S. Swaroopa, R. Mukherjee, A. Debnath, and R. S. Chakraborty, “Evaluating large language models for automatic register transfer logic generation via high-level synthesis,” arXiv preprint arXiv:2408.02793 , 2024

2024 arXiv
[33]

Benchmarking large language models for auto- mated verilog rtl code generation,

S. Thakur, B. Ahmad, Z. Fan, H. Pearce, B. Tan, R. Karri, B. Dolan- Gavitt, and S. Garg, “Benchmarking large language models for auto- mated verilog rtl code generation,” in 2023 Design, Automation & Test in Europe Conference & Exhibition (DATE) . IEEE, 2023, pp. 1–6

2023
[34]

Verigen: A large language model for verilog code generation,

S. Thakur, B. Ahmad, H. Pearce, B. Tan, B. Dolan-Gavitt, R. Karri, and S. Garg, “Verigen: A large language model for verilog code generation,” ACM Transactions on Design Automation of Electronic Systems, vol. 29, no. 3, pp. 1–31, 2024

2024
[35]

Vhdl-eval: A framework for evaluating large language models in vhdl code generation,

P. Vijayaraghavan, L. Shi, S. Ambrogio, C. Mackin, A. Nitsure, D. Beymer, and E. Degan, “Vhdl-eval: A framework for evaluating large language models in vhdl code generation,” 2024. [Online]. Available: https://arxiv.org/abs/2406.04379

2024 arXiv
[36]

Understanding the i2c bus: A beginner’s guide to simplifying communication,

P. Waite, “Understanding the i2c bus: A beginner’s guide to simplifying communication,” https://wraycastle.com/blogs/knowledge- base/understanding-the-i2c-bus-a-beginners-guide-to-simplifying- communication, 2024, accessed: 16th April 2025

2024
[37]

Vflow: Discovering optimal agentic workflows for verilog generation,

Y . Wei, Z. Huang, H. Li, W. W. Xing, T.-J. Lin, and L. He, “Vflow: Discovering optimal agentic workflows for verilog generation,” arXiv preprint arXiv:2504.03723, 2025

2025 arXiv
[38]

Verithoughts: Enabling automated verilog code generation us- ing reasoning and formal verification,

P. Yubeaton, A. Nakkab, W. Xiao, L. Collini, R. Karri, C. Hegde, and S. Garg, “Verithoughts: Enabling automated verilog code generation us- ing reasoning and formal verification,” arXiv preprint arXiv:2505.20302, 2025

2025
[2023]

Available: https://arxiv.org/abs/2307.03817

[Online]. Available: https://arxiv.org/abs/2307.03817

arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Claude opus

“Claude opus.” [Online]. Available: https://www.anthropic.com/claude/ opus

work page

[2] [2]

Claude sonnet

“Claude sonnet.” [Online]. Available: https://www.anthropic.com/claude/ sonnet

work page

[3] [3]

ARM, Introduction to AMBA AXI4 , ARM limited, 2020

work page 2020

[4] [4]

Pyhdl- eval: An llm evaluation framework for hardware design using python- embedded dsls,

C. Batten, N. Pinckney, M. Liu, H. Ren, and B. Khailany, “Pyhdl- eval: An llm evaluation framework for hardware design using python- embedded dsls,” in 2024 ACM/IEEE 6th Symposium on Machine Learn- ing for CAD (MLCAD) , 2024, pp. 1–17

work page 2024

[5] [5]

Smart communication in a wired sensor- and actuator-network of a modular robot actuator system using a hop- protocol with delta-routing,

S. Bosse and D. Lehmhus, “Smart communication in a wired sensor- and actuator-network of a modular robot actuator system using a hop- protocol with delta-routing,” in Proceedings of smart systems integration conference, Como, Italy , 2010, pp. 23–24

work page 2010

[6] [6]

Chipgpt: How far are we from natural language hardware design,

K. Chang, Y . Wang, H. Ren, M. Wang, S. Liang, Y . Han, H. Li, and X. Li, “Chipgpt: How far are we from natural language hardware design,” arXiv preprint arXiv:2305.14019 , 2023

arXiv 2023

[7] [7]

Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities,

G. Comanici, E. Bieber, M. Schaekermann et al., “Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities,” 2025. [Online]. Available: https://arxiv.org/abs/2507.06261

arXiv 2025

[8] [8]

Exploring and characterizing large language models for embedded system development and debugging,

Z. Englhardt, R. Li, D. Nissanka, Z. Zhang, G. Narayanswamy, J. Breda, X. Liu, S. Patel, and V . Iyer, “Exploring and characterizing large language models for embedded system development and debugging,”

work page