Pith. sign in

REVIEW 4 major objections 4 minor 2 cited by

LASHED: LLMs And Static Hardware Analysis for Early Detection of RTL Bugs

T0 review · 4 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A pipeline that pairs a large language model with static analysis can flag plausible hardware security bugs in RTL code with 87.5 percent precision.

desk verdict Useful pipeline and prompt analysis, but the 87.5% precision claim is not independently validated; send to review with a request for harder ground truth. read the letter →

arxiv 2504.21770 v1 pith:EHCJHKOK submitted 2025-04-30 cs.CR

Baleegh Ahmad , Hammond Pearce , Ramesh Karri , Benjamin Tan This is my paper · ORCID
classification cs.CR
keywords hardwaresecurityRTLbugdetectionstaticanalysislargelanguagemodelsCommonWeaknessEnumerationSystemVerilogassertionsRISC-VSoCsin-contextlearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

LASHED is a proposed three-step pipeline for detecting security bugs in register-transfer-level (RTL) hardware designs, combining a large language model with static analysis so that each covers the other's blind spots. The LLM first names security-relevant signals in the code; a linter or formal property checker then turns those signals into concrete checks; the LLM finally prunes false alarms and writes a plain-language explanation for each remaining issue. The paper's central result is that on four open-source RISC-V system-on-chips, across five hardware Common Weakness Enumerations, the recommended configuration (a stronger LLM with an in-context example and a "think again" step) flagged 40 instances, 35 of which the authors judged to be plausible CWEs, a precision of 87.5%. The authors argue this makes early, explainable security screening feasible without requiring a full formal-verification setup for every check.

What carries the argument

The engine of LASHED is the three-stage division of labor it calls AssetID, Static Analysis, and Contextualization. In AssetID, the LLM is given RTL source plus a hardware CWE description and asked to list security-relevant signals, such as lock bits, privilege levels, or debug-access passwords. Static Analysis then uses those signals to focus an existing tool: for structural weaknesses it runs lint checks on assignments and conditionals, and for behavioral weaknesses it fills custom SystemVerilog assertion templates—asserting that lock signals stay stable, that protected registers change only under proper lock, and that privilege escalation obeys its conditions—and checks them with formal property verification. Contextualization sends the lint violations and falsified assertions back to the LLM, which decides which ones are genuine security issues and explains them. The key mechanism is that the LLM supplies the semantic context static tools lack, while the static tools supply the verifiable evidence LLMs do not generate on their own.

What would settle it

Run the recommended LASHED configuration blind on a clean open-source RISC-V design into which known instances of CWE-1191, CWE-1231, CWE-1233, CWE-1244, and CWE-1300 have been injected, then compare its flagged locations against the injection list; if substantially fewer than 87.5% of its flags match independently known bugs, the reported precision depends on the authors' labeling rather than on the method.

Watch

Extended reading notes

Core claim

The central claim is that combining an LLM with static analysis detects real hardware security weaknesses in RTL code that neither tool reliably finds alone, and that the combination can be steered by prompt engineering. In the paper's evaluation, a total of 545 instances were flagged across 160 experiments, of which 51% were considered plausible CWEs; precision varied by weakness, from 100% for improper debug access control (CWE-1191) down to 28% for improper lock-bit protection (CWE-1231). The recommended scheme, gpt-4o with prompt variation v3, achieved 87.5% precision: 35 of 40 flagged instances. The paper also reports that in-context examples mainly improve asset identification, asking the model to "think again" mainly removes false positives, and the stronger model produces fewer false positives overall.

Load-bearing premise

The load-bearing premise is that the authors' manual review of flagged violations is a valid ground truth; the headline precision comes from those labels and from a configuration chosen after inspecting the same results, so biased labeling or selection would overstate the claim.

Editorial extensions

If this is right

  • For a given CWE and SoC, LASHED reports only a handful of candidate issues (3.4 on average), each with a source line and explanation, instead of drowning the designer in raw lint warnings.
  • Design teams can apply the same prompts to large, unfamiliar designs: the approach was run on a root-of-trust chip of about 171k lines of code, not just the small benchmark used for prototyping.
  • Prompt design is a first-class knob: adding one in-context example and a second look at contextualization raised precision from 44% to 62% across all runs, and 87.5% in the recommended configuration.
  • The two static-analysis styles work for different weakness families, so a practical deployment would choose linting for structural or side-channel checks and assertions for lock-bit and privilege checks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension beyond the paper's five CWEs would be to test whether the same two-template style scales to other hardware weakness classes; the paper itself notes that CWE-1233 was hardest because protected registers take many forms, so new templates would likely be needed.
  • A more decisive evaluation would use injected-bug benchmarks to measure recall, not just precision; the paper currently can only confirm bugs after flagging them, so it never measures how many real bugs were missed.
  • Given that the v1 examples were drawn from one of the evaluated SoCs, I would expect precision on a completely unrelated CWE-specific benchmark to be lower; measuring that drop would separate in-context learning from memorization.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes LASHED, a three-stage pipeline for early detection of RTL hardware security bugs: an LLM identifies CWE-relevant assets; a static analysis stage (either lint checks or assertion-based formal verification, depending on the CWE) flags violations; and a second LLM pass prunes false positives and provides explanations. The authors evaluate LASHED on four open-source RISC-V SoCs across five CWEs, four prompt variations (v0-v3), and two LLMs (gpt-4o-mini and gpt-4o), reporting aggregate precision and a headline claim that the recommended configuration (gpt-4o with variation v3) achieves 87.5% precision (35 of 40 flagged instances). They also analyze the effect of in-context learning and self-reflection prompts on precision.

Significance. The direction is timely and potentially useful: combining static analysis with LLM contextualization for hardware security checking is under-explored, and the paper provides a concrete instantiation with explicit CWE-to-lint/assertion mappings and a multi-SoC study. If the headline precision were robust, this would be a meaningful step toward practical pre-silicon security screening. The paper also deserves credit for reporting the aggregate 50.8% precision and per-CWE breakdowns, which makes the variability across CWEs visible. However, the central quantitative claim is currently supported only by author-confirmed labels, post-hoc configuration selection, and a prompt that is partly derived from one of the evaluation SoCs, so the evidence is not yet convincing.

major comments (4)
  1. [Section IV and Table III] The 87.5% headline is the maximum of eight experimental configurations (2 LLMs x 4 prompt variations) and is selected after inspecting the results on the same data. The paper's own aggregate numbers are much lower: 277/545 = 50.8% overall, and the best average variation precision is 62% for v3. No confidence intervals or significance tests are reported; for the 40-flagged-instance cell behind the headline, the 95% Wilson interval spans roughly [74%, 95%], so the point estimate is compatible with substantially lower precision. The claim that in-context learning and 'thinking again' improve precision needs a pre-registered or held-out selection procedure and a statistical comparison.
  2. [Section IV-A2, Section III-C, and Appendix B] The paper states in Section IV-A2 that the v1 guiding examples are 'inspired from the Hack@DAC-21 SoC', and v3 includes v1, yet Hack@DAC-21 is one of the four evaluation SoCs. This is a test-set leakage for the recommended prompt. The contradiction with Section III-C, which says the v1 examples are taken from the MITRE website, must be resolved. Moreover, Section III-B says the CWEs were selected because they appear in Hack@DAC-21 'to have some ground-truth', but the reported precision never uses that ground truth: the paper does not check whether LASHED flags the known injected bugs. As a result, the evaluation is circular for one SoC and unvalidated for the other three.
  3. [Section IV and Section V] The true-positive counts are 'author-confirmed violations' established by visual inspection (Section V), with no second annotator, no inter-annotator agreement statistic, and no independent benchmark for OpenTitan, E203, or Veerwolf. The authors explicitly acknowledge this limitation, but the abstract and conclusion still present 87.5% as an empirical result. Since precision is the only performance metric reported, the headline should be reframed as 'author-validated precision' and corroborated by external labels or at minimum a blinded second annotation.
  4. [Section IV and Appendix A] Each experiment appears to be a single run with no control for LLM stochasticity (temperature, seed, or repeated calls), and some cells have very small counts (e.g., CWE 1244 flagged cells with 1-3 instances). The quantitative comparisons between prompt variations and between LLMs therefore rest on unstable point estimates; repeated runs or confidence intervals are needed before drawing conclusions.
minor comments (4)
  1. [Throughout] Typos: 'Identificaiton' in Section II-A, 'to conduct out experiments' in Section III-D, 'gpt-40-mini' in Section IV-A3, 'Propmpt' in Appendix B, and 'time∝ loc0.53 approximately ... produces follows the relation' in Appendix A.
  2. [Table III] Table III defines FDR as FP/Flagged, which is not the standard false discovery rate; use 'false positive proportion' or '1 - precision' instead.
  3. [Figures 4 and 5] Figures 4 and 5 use the label 'Variation(LLM)' and do not show numerical values on the stacked bars, making it difficult to verify the totals in Table III from the figures.
  4. [Section II-B] The code snippets in Section II (e.g., the assertion templates with 'd i s a b l e i f f') contain spacing artifacts from the PDF rendering; ensure these are formatted correctly in the final version.

Circularity Check

3 steps flagged · score 7.0 of 10

Headline 87.5% precision is an in-sample, best-of-eight result whose true positives are author-confirmed judgments; there is no independent ground truth for three of four SoCs, and the recommended v3 prompt draws examples from an evaluated SoC.

  1. fitted input called prediction [Section IV (Results), Table III; Section III-C (Prompt Variations)]
    "For our recommended combination of using gpt-4o with v3, 35 of the 40 flagged instances are plausible CWEs, providing a precision of 87.5%. ... We evaluated the violations manually, with our author-confirmed violations providing the True Positives count in Table III."

    The 87.5% figure is not an out-of-sample prediction: v3 was selected as the best of the 2 x 4 = 8 configurations after all results were available (Table III marks the best results in bold), and the true-positive numerator is the authors' own manual confirmation of the same flagged instances. Aggregate precision is 277/545 = 50.8%, and the best variation-level precision is 62%, so the headline number is an in-sample selected optimum. Reporting it as the precision of the recommended scheme is statistically forced by the selection and labeling procedure, not independently validated.

  2. self definitional [Section IV (Results); Section V (Discussion)]
    "Since we do not know the real number of bugs (we can only confirm or deny a specific bug’s presence after flagging), it is not possible to calculate a proper Recall or Accuracy score. Therefore, we rely on Precision i.e. (# true positives / # predicted positive (flagged)) as the metric to evaluate performance. We evaluated the violations manually, with our author-confirmed violations providing the True Positives count in Table III."

    The 'true positive' set is defined as the authors' manual confirmation of flagged instances, so precision measures agreement between LASHED and the authors' plausibility judgment, not detection against an independent ground truth. For three of the four SoCs there is no known-bug benchmark, and the paper concedes in Section V: 'Another limitation lies in the manual evaluation of flagged violations. We assessed all violations through visual inspection which has a possibility of being incorrect.' The central 87.5% claim therefore reduces by construction to author-system agreement, not to independently known bugs.

1 more flagged steps
  1. other [Section IV-A2 (Which SoCs were better analyzed?)]
    "Hack@DAC-21 performs the second best with a precision of 0.55, which is expected because the guiding examples we use in v1 are inspired from the Hack@DAC-21 SoC."

    The recommended v3 includes v1, and v1's in-context examples are drawn (or 'inspired') from Hack@DAC-21, one of the four evaluated SoCs. The paper itself attributes that SoC's better precision to this overlap. This is test-set leakage rather than a formal derivation reduction, but it further undermines the independence of the reported 87.5% for the recommended prompt, because the prompt was partially tuned on an evaluated test item.

full rationale

The paper's framework itself is a legitimate combination of LLM asset identification, static analysis, and LLM contextualization, and the authors are transparent that they cannot compute recall and rely on precision. The circularity is in the evaluation of the central quantitative claim. The paper explicitly defines true positives as 'author-confirmed violations' and selects the best of eight prompt/LLM configurations after inspecting the same results, so the headline 35/40 = 87.5% is an in-sample selected optimum over manually labeled instances, not an independent prediction. The v1 in-context examples being inspired by Hack@DAC-21, one of the evaluated SoCs, adds contamination. No load-bearing self-citation chain occurs here: references to the authors' prior work [9], [11], [22], [23] are contextual and do not force the result. The score reflects that the headline precision reduces, by the paper's own definitions and selection procedure, to author-system agreement rather than validated detection.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

This is an empirical systems paper, so the ledger records hand-selected design choices and domain assumptions rather than fitted mathematical constants. The central quantitative claim depends on the authors' manual labels and on post-hoc selection of the prompt variation, which are the weakest links.

free parameters (4)
  • Per-CWE strategy choice (lint vs. assertion)
    Authors choose linting for CWEs 1191 and 1300 and assertion templates for CWEs 1231, 1233, and 1244 by hand; this choice determines which static analysis is applied and therefore affects all results.
  • Prompt variation components (v1 example and v2 self-reflection) = v3 for recommended scheme
    The v1 in-context example is taken from MITRE and Hack@DAC-21; the v2 instruction asks the model to reconsider issues only when confident. These are hand-designed and selected post hoc; the recommended scheme is the best of all variations on the test data.
  • Lint check tags selected for each CWE
    For CWE 1191, six lint tags (width mismatch, concatenation patterns, and similar) are selected; for CWE 1300, 'if without else' and 'inferred latches' are selected. These hand-chosen checks encode the authors' understanding of the CWE.
  • Assertion templates for CWEs 1231, 1233, 1244
    The SystemVerilog Assertion templates are written by the authors and populated with LLM-identified assets; correctness of template and asset conditions is a manual input.
assumptions (4)
  • domain assumption The dataset of four open-source RISC-V SoCs is representative of real RTL designs
    Evaluation is limited to these four designs; generalization to other designs, ASICs, or other ISAs is assumed.
  • domain assumption Manual author labeling of true positives is a valid ground truth
    Precision is the only metric and references the authors' visual inspection; no independent annotator or confirmed bug list exists for OpenTitan, E203, and VeerWolf.
  • ad hoc to paper The CWE to static-analysis mapping (lint tags and assertion templates) captures the weakness class
    For example, CWE 1231 is expressed as stability of lock signals through an SVA template; if this modeling is wrong, the assertions flag non-issues and miss real bugs.
  • domain assumption LLM and tool outputs are stable enough for single-run measurements to be meaningful
    Each experiment is run once with no error bars; the stochasticity of LLM responses is not quantified.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LASHED: LLMs And Static Hardware Analysis for Early Detection of RTL Bugs." pith.science (2026). https://pith.science/paper/EHCJHKOK

@misc{pith2026250421770,
  author       = {Pith},
  title        = {Pith review of: LASHED: LLMs And Static Hardware Analysis for Early Detection of RTL Bugs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EHCJHKOK}},
  note         = {Machine review of arXiv:2504.21770}
}
read the original abstract

While static analysis is useful in detecting early-stage hardware security bugs, its efficacy is limited because it requires information to form checks and is often unable to explain the security impact of a detected vulnerability. Large Language Models can be useful in filling these gaps by identifying relevant assets, removing false violations flagged by static analysis tools, and explaining the reported violations. LASHED combines the two approaches (LLMs and Static Analysis) to overcome each other's limitations for hardware security bug detection. We investigate our approach on four open-source SoCs for five Common Weakness Enumerations (CWEs) and present strategies for improvement with better prompt engineering. We find that 87.5% of instances flagged by our recommended scheme are plausible CWEs. In-context learning and asking the model to 'think again' improves LASHED's precision.

Figures

Figures reproduced from arXiv: 2504.21770 by the authors.

Figure 1
Figure 1. LASHED framework. 1) Assets Identification (AssetID): LLM identifies assets relevant to a CWE in the RTL. 2) Static Analysis: Depending on CWE, a linting or assertion-based strategy is used. A linter or formal property verification tool checks for the lint tags identified or assertions formed. 3) Contextualization: LLM reasons and prunes linting violations or falsified properties. LLM gives an explanation if there i… view at source ↗
Figure 2
Figure 2. Motivating example for CWE 1191 to illustrate LASHED flow for [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 4
Figure 4. Classification of instances flagged by LASHED. Stacked bar shows the number of instances identified by LASHED that pose security issues as true [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figures from the paper (1 more)
Figure 5
Figure 5. Figure 5: Intermediary outputs during LASHED’s operation. # assets identified is number of security relevant signals identified during Assets Identification, # [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. CHARGE: Leveraging CWE Hierarchies for Hardware Security SystemVerilog Assertion Generation

    cs.CR 2026-07 conditional novelty 6.0 of 10

    Using CWE hierarchy-aware LLM prompts, CHARGE generates security SVAs from unverified RTL, detecting 27 of 42 Hack@DAC bugs and one new key-reuse flaw.

  2. Hardware Design and Security in the Era of Chiplets and LLMs

    cs.CR 2026-08 accept novelty 2.0 of 10

    A survey uniting chiplet-hardware security and LLM-driven EDA security that identifies a missing bridge: LLM-based security tools are not yet tailored to 2.5D/3D chiplet systems.

Reference graph

Works this paper leans on

38 extracted references · 29 canonical work pages · cited by 2 Pith papers

  1. [1]

    HardFails: Insights into Software-Exploitable Hardware Bugs,

    G. Dessouky, D. Gens, P. Haney, G. Persyn, A. Kanuparthi, H. Khattri, J. M. Fung, A.-R. Sadeghi, and J. Rajendran, “HardFails: Insights into Software-Exploitable Hardware Bugs,” 2019, pp. 213–230. [Online]. Available: https://www.usenix.org/conference/ usenixsecurity19/presentation/dessouky

    work page 2019

  2. [2]

    Post-silicon validation opportunities, challenges and recent advances,

    S. Mitra, S. A. Seshia, and N. Nicolici, “Post-silicon validation opportunities, challenges and recent advances,” in Proceedings of the 47th Design Automation Conference , ser. DAC ’10. New York, NY , USA: Association for Computing Machinery, Jun. 2010, pp. 12–17. [Online]. Available: https://dl.acm.org/doi/10.1145/1837274.1837280

    work page arXiv

  3. [3]

    INVITED: Formal Verification of Security Critical Hardware-Firmware Interactions in Commercial SoCs,

    S. Ray, N. Ghosh, R. Masti, A. Kanuparthi, and J. Fung, “INVITED: Formal Verification of Security Critical Hardware-Firmware Interactions in Commercial SoCs,” in 2019 56th ACM/IEEE Design Automation Conference (DAC), Jun. 2019, pp. 1–4, iSSN: 0738-100X

    work page 2019

  4. [4]

    SoC interconnection protection through formal verification,

    J. He, X. Guo, T. Meade, R. Dutta, Y . Zhao, and Y . Jin, “SoC interconnection protection through formal verification,” Integration, vol. 64, pp. 143–151, Jan. 2019. [Online]. Available: https://www. sciencedirect.com/science/article/pii/S016792601830289X

    work page 2019

  5. [5]

    DifuzzRTL: Differential Fuzz Testing to Find CPU Bugs,

    J. Hur, S. Song, D. Kwon, E. Baek, J. Kim, and B. Lee, “DifuzzRTL: Differential Fuzz Testing to Find CPU Bugs,” in 2021 IEEE Symposium on Security and Privacy (SP) , May 2021, pp. 1286–1303, iSSN: 2375- 1207

    work page 2021

  6. [6]

    Fuzzing Hardware Like Software,

    T. Trippel, K. G. Shin, A. Chernyakhovsky, G. Kelly, D. Rizzo, and M. Hicks, “Fuzzing Hardware Like Software,” 2022, pp. 3237–3254. [Online]. Available: https://www.usenix.org/conference/ usenixsecurity22/presentation/trippel

    work page 2022

  7. [7]

    Register transfer level information flow tracking for provably secure hardware design,

    A. Ardeshiricham, W. Hu, J. Marxen, and R. Kastner, “Register transfer level information flow tracking for provably secure hardware design,” in Design, Automation Test in Europe Conference Exhibition (DATE), 2017, Mar. 2017, pp. 1691–1696, iSSN: 1558-1101

    work page 2017

  8. [8]

    Hardware Information Flow Tracking,

    W. Hu, A. Ardeshiricham, and R. Kastner, “Hardware Information Flow Tracking,” ACM Computing Surveys , vol. 54, no. 4, pp. 83:1–83:39, May 2021. [Online]. Available: https://dl.acm.org/doi/10.1145/3447867

    doi:10.1145/3447867 2021

Show all 38 references
  1. [9]

    Don’t CWEAT It: Toward CWE Analysis Techniques in Early Stages of Hardware Design,

    B. Ahmad, W.-K. Liu, L. Collini, H. Pearce, J. M. Fung, J. Valamehr, M. Bidmeshki, P. Sapiecha, S. Brown, K. Chakrabarty, R. Karri, and B. Tan, “Don’t CWEAT It: Toward CWE Analysis Techniques in Early Stages of Hardware Design,” in Proceedings of the 41st IEEE/ACM Internationa...

    2022

  2. [10]

    Hunting Security Bugs in SoC Designs: Lessons Learned,

    M. M. Bidmeshki, Y . Zhang, M. Zaman, L. Zhou, and Y . Makris, “Hunting Security Bugs in SoC Designs: Lessons Learned,” IEEE Design & Test, vol. 38, no. 1, pp. 22–29, Feb. 2021

    2021

  3. [11]

    FLAG: Finding Line Anomalies (in code) with Generative AI,

    B. Ahmad, B. Tan, R. Karri, and H. Pearce, “FLAG: Finding Line Anomalies (in code) with Generative AI,” Jun. 2023, arXiv:2306.12643 [cs]. [Online]. Available: http://arxiv.org/abs/2306.12643

    2023 arXiv

  4. [12]

    SoCureLLM: An LLM-driven Approach for Large-Scale System-on- Chip Security Verification and Policy Generation,

    S. Tarek, D. Saha, S. K. Saha, M. Tehranipoor, and F. Farahmandi, “SoCureLLM: An LLM-driven Approach for Large-Scale System-on- Chip Security Verification and Policy Generation,” 2024, publication info: Preprint. [Online]. Available: https://eprint.iacr.org/2024/983

    2024

  5. [13]

    Self-HWDebug: Automation of LLM Self-Instructing for Hardware Security Verification,

    M. Akyash and H. M. Kamali, “Self-HWDebug: Automation of LLM Self-Instructing for Hardware Security Verification,” May 2024, arXiv:2405.12347. [Online]. Available: http://arxiv.org/abs/2405.12347

    2024 arXiv

  6. [14]

    Synopsys VC SpyGlass Lint,

    V . S. Lint, “Synopsys VC SpyGlass Lint,”

  7. [15]

    Jasper Superlint App,

    jasperlint, “Jasper Superlint App,” 2022. [Online]. Available: https://www.cadence.com/en_US/home/tools/ system-design-and-verification/formal-and-static-verification/ jasper-gold-verification-platform/jaspergold-superlint-app.html

    2022

  8. [16]

    VC Formal,

    “VC Formal,” 2022. [Online]. Available: https://www.synopsys.com/ verification/static-and-formal-verification/vc-formal.html

    2022

  9. [17]

    Jasper RTL Apps | Cadence,

    Cadence, “Jasper RTL Apps | Cadence,” Jul. 2022. [Online]. Available: https://www.cadence.com/en_US/home/ tools/system-design-and-verification/formal-and-static-verification/ jasper-gold-verification-platform.html

    2022

  10. [18]

    Continuous Linting with Automatic Debug,

    D. Hansson, “Continuous Linting with Automatic Debug,” in 2014 15th International Microprocessor Test and Verification Workshop, Dec. 2014, pp. 70–72, iSSN: 2332-5674

    2014

  11. [19]

    Linting for Visualization: Towards a Practical Automated Visualization Guidance System,

    A. McNutt and G. Kindlmann, “Linting for Visualization: Towards a Practical Automated Visualization Guidance System,” 2018

    2018

  12. [20]

    Formal methods: Practice and experience,

    J. Woodcock, P. G. Larsen, J. Bicarregui, and J. Fitzgerald, “Formal methods: Practice and experience,” ACM Comput. Surv. , vol. 41, no. 4, pp. 19:1–19:36, Oct. 2009. [Online]. Available: https://dl.acm.org/doi/10.1145/1592434.1592436

    2009

  13. [21]

    Evaluating Large Language Models Trained on Code,

    M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. d. O. Pinto, J. Kaplan, H. Edwards, Y . Burda, N. Joseph, G. Brockman, A. Ray, R. Puri, G. Krueger, M. Petrov, H. Khlaaf, G. Sastry, P. Mishkin, B. Chan, S. Gray, N. Ryder, M. Pavlov, A. Power, L. Kaiser, M. Bavarian, C. Winter, P. Ti...

    2021 arXiv

  14. [22]

    VeriGen: A Large Language Model for Verilog Code Generation,

    S. Thakur, B. Ahmad, H. Pearce, B. Tan, B. Dolan-Gavitt, R. Karri, and S. Garg, “VeriGen: A Large Language Model for Verilog Code Generation,” ACM Trans. Des. Autom. Electron. Syst. , vol. 29, no. 3, pp. 46:1–46:31, Apr. 2024. [Online]. Available: https://doi.org/10.1145/3643681

    2024 doi

  15. [23]

    On Hardware Security Bug Code Fixes by Prompting Large Language Models,

    B. Ahmad, S. Thakur, B. Tan, R. Karri, and H. Pearce, “On Hardware Security Bug Code Fixes by Prompting Large Language Models,” IEEE Transactions on Information Forensics and Security , vol. 19, pp. 4043–4057, 2024. [Online]. Available: https://ieeexplore.ieee.org/ abstract/do...

    2024

  16. [24]

    LLM-Assisted Static Analysis for Detecting Security Vulnerabilities,

    Z. Li, S. Dutta, and M. Naik, “LLM-Assisted Static Analysis for Detecting Security Vulnerabilities,” Nov. 2024, arXiv:2405.17238. [Online]. Available: http://arxiv.org/abs/2405.17238

    2024 arXiv

  17. [25]

    Interleaving Static Analysis and LLM Prompting,

    P. J. Chapman, C. Rubio-González, and A. V . Thakur, “Interleaving Static Analysis and LLM Prompting,” in Proceedings of the 13th ACM SIGPLAN International Workshop on the State Of the Art in Program Analysis. Copenhagen Denmark: ACM, Jun. 2024, pp. 9–17. [Online]. Available: ...

    2024

  18. [26]

    Enhancing Static Analysis for Practical Bug Detection: An LLM-Integrated Approach,

    H. Li, Y . Hao, Y . Zhai, and Z. Qian, “Enhancing Static Analysis for Practical Bug Detection: An LLM-Integrated Approach,” Enhancing Static Analysis for Practical Bug Detection: An LLM-Integrated Approach (Artifact), vol. 8, no. OOPSLA1, pp. 111:474–111:499, Apr

  19. [27]

    CWE - CWE Most Important Hardware Weaknesses,

    T. M. C. (MITRE), “CWE - CWE Most Important Hardware Weaknesses,” 2022. [Online]. Available: https://cwe.mitre.org/scoring/ lists/2021_CWE_MIHW.html

    2022

  20. [28]

    Verific Design Automation,

    Verific, “Verific Design Automation,” 2022. [Online]. Available: https://www.verific.com/

    2022

  21. [29]

    VC Formal: Formal Verification Solution | Synopsys,

    V . Formal, “VC Formal: Formal Verification Solution | Synopsys,”

  22. [30]

    HACK-EVENT/hackatdac21,

    “HACK-EVENT/hackatdac21,” Apr. 2024, original-date: 2023-07- 15T20:58:02Z. [Online]. Available: https://github.com/HACK-EVENT/ hackatdac21

    2024

  23. [31]

    Open source silicon root of trust (RoT) | OpenTitan,

    lowRISC contributors, “Open source silicon root of trust (RoT) | OpenTitan,” 2023. [Online]. Available: https://opentitan.org/

    2023

  24. [32]

    Available: https://www.synopsys.com/verification/ static-and-formal-verification/vc-formal.html

    [Online]. Available: https://www.synopsys.com/verification/ static-and-formal-verification/vc-formal.html

  25. [33]

    VeeRwolf,

    chipsalliance, “VeeRwolf,” Nov. 2024, original-date: 2019-08- 07T15:24:36Z. [Online]. Available: https://github.com/chipsalliance/ VeeRwolf

    2024

  26. [34]

    Large Language Models Are Human-Level Prompt Engineers,

    Y . Zhou, A. I. Muresanu, Z. Han, K. Paster, S. Pitis, H. Chan, and J. Ba, “Large Language Models Are Human-Level Prompt Engineers,” Nov. 2022, arXiv:2211.01910 [cs]. [Online]. Available: http://arxiv.org/abs/2211.01910

    2022 arXiv

  27. [35]

    Hummingbirdv2 E203 Core and SoC - GitHub,

    N. S. Technology, “Hummingbirdv2 E203 Core and SoC - GitHub,” May 2022, original-date: 2020-07-29T06:28:49Z. [Online]. Available: https://github.com/riscv-mcu/e203_hbirdv2

    2022

  28. [38]

    "" always @(posedge clk_i) begin if( (rst_ni && jtag_unlock && rst_9)) begin for (j=0; j < 6; j=j+1) begin reglk_mem[j] <= ’h0

    OpenAI, “GPT-4o,” May 2024. [Online]. Available: https://openai.com/ index/hello-gpt-4o/ APPENDIX A. Scalability and Cost On average, each experiment took 163 seconds to run. This time includes the complete flow of running the LLM and static analysis tools from the identificat...

    2024

  29. [2022]

    Available: https://www.synopsys.com/verification/ static-and-formal-verification/vc-spyglass/vc-spyglass-lint.html

    [Online]. Available: https://www.synopsys.com/verification/ static-and-formal-verification/vc-spyglass/vc-spyglass-lint.html

  30. [2024]

    Available: https://dl.acm.org/doi/10.1145/3649828

    [Online]. Available: https://dl.acm.org/doi/10.1145/3649828

    doi

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.