Pith. sign in

REVIEW 5 major objections 5 minor 18 references

BugWhisperer: Fine-Tuning LLMs for SoC Hardware Vulnerability Detection

T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Pith's one-line read Fine-tuning an open-source LLM on a curated database of vulnerable RTL designs lifts hardware vulnerability detection accuracy by over 40 percentage points.

desk verdict Useful open artifact—fine-tuned Mistral and a GPT-generated RTL vulnerability database—but the headline accuracy is measured against GPT-generated ground truth and needs an independent check before the claims hold. read the letter →

arxiv 2505.22878 v1 pith:I2WYFVBM submitted 2025-05-28 cs.CR cs.AI

classification cs.CRcs.AI
keywords hardwaresecurityLLMfine-tuningSoCvulnerabilitydetectionRTLverificationVerilogCWEdatabaseLoRAopen-source
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

BugWhisperer aims to show that a small, open-source language model, fine-tuned on a purpose-built database of vulnerable and secure register-transfer level (RTL) designs, can detect system-on-chip (SoC) security vulnerabilities almost as well as proprietary models. The authors build the database by taking 13 known vulnerability classes from existing SoC benchmarks and using GPT-4 and GPT-4o to generate 4,000 varied Verilog replicas plus explanatory annotations. After instruction fine-tuning with low-rank adaptation, the best model, Mistral-7B-instruct, reaches 84.8% detection accuracy versus 42.5% for the same model unmodified, a gain of over 40 percentage points, with GPT-4o at 91.3%. The paper's point is that domain-adapted open models are a viable, cost-effective alternative to closed commercial APIs for hardware security verification.

What carries the argument

The carrying mechanism is an instruction fine-tuning pipeline centered on a curated hardware vulnerability database. Seed designs with 13 labeled CWE (Common Weakness Enumeration) vulnerability classes are replicated into 4,000 distinct Verilog implementations by GPT-4/GPT-4o under varied coding styles (parameterization, single- and dual-process FSMs, renamed signals), with GPT-4o supplying explanatory annotations that become the model's reasoning targets. Each fine-tuned model is trained with low-rank adaptation (LoRA, rank 128, alpha 256) on prompts that ask whether a given design contains a specified vulnerability, with responses stating presence or absence and justifying the answer. The test split is partitioned by design, so the reported accuracy measures generalization to RTL code the model never saw during training.

What would settle it

Take a random sample of the 4,000 generated designs and check each one formally or by simulation against its stated CWE behavior; if a substantial fraction fail to exhibit the injected vulnerability, the reported 84.8% accuracy is an overestimate of real detection ability.

Watch

Extended reading notes

Core claim

The central claim is that fine-tuning on hardware-specific vulnerability data transforms a general-purpose instruction-tuned LLM into a competent RTL security auditor. Evaluating on 13 CWE-defined vulnerabilities across an unseen test split of SoC modules, the fine-tuned Mistral-7B-instruct detects 84.8% of vulnerabilities, outperforming its non-fine-tuned version by more than 40 percentage points and approaching GPT-4o's 91.3%. Similar gains appear for the fine-tuned Llama-3.2-3B (68.7%) and Llama-3.1-8B (74.4%) models, while non-fine-tuned open models average around 40%. The paper interprets this as evidence that the limiting factor for open-source LLMs in hardware security is domain knowledge rather than model capacity, and that deliberate dataset construction plus fine-tuning can close most of the gap to proprietary systems.

Load-bearing premise

The accuracy numbers rest on an unverified assumption: that the 4,000 GPT-generated Verilog replicas still contain exactly the CWE vulnerability they are labeled with, because no simulation, formal check, or manual audit was run on them.

Editorial extensions

If this is right

  • RTL security screening can be automated early in the SoC design flow, catching flaws before fabrication at far lower cost than manual review.
  • Open-source models fine-tuned on this database become practical for organizations that cannot expose designs to proprietary cloud APIs.
  • The released 4,000-design vulnerability database gives other teams a common benchmark for training and measuring hardware security models.
  • The consistent gains across five base models suggest the method transfers to other open instruction-tuned LLMs, not just the three reported.
  • Detection at 68–85% still leaves a meaningful miss rate, so the approach is a screening tool rather than a replacement for formal verification.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Not stated in the paper: a natural next test is whether the accuracy holds on the original, non-replicated seed modules or on freshly written RTL from other cores, since the current evaluation only measures performance on GPT-generated variants of the same seed designs.
  • Not stated in the paper: if the GPT-generated labels are noisy, the reported margins may partly reflect the model learning the Replicator LLM's style rather than the vulnerability itself; an independent audit of a sample of the 4,000 replicas would settle this.
  • Not stated in the paper: the pipeline could extend beyond the 13 CWEs to privilege escalation, side-channel, and fault-injection classes, but each new class would need its own gold benchmarks and replicated instances.
  • Not stated in the paper: because non-fine-tuned baselines were prompted without the curated explanations, part of the gain may come from better prompts and label style rather than from new hardware knowledge; an ablation with identical prompts would separate these effects.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes BugWhisperer, a framework that creates 4,000 Verilog/SystemVerilog replicas of 13 vulnerable modules from the Cad4Security Ariane SoC benchmarks using GPT-4/GPT-4o, augments them with GPT-4o-written explanatory annotations, and fine-tunes open-source LLMs (Llama-3.2-1B/3B, Llama-3.1-8B, Mistral-7B, CodeLlama-7B) with LoRA for 13 CWE-class vulnerability detection. Evaluation reports a single accuracy value per model, with fine-tuned Mistral-7B reaching 84.8% versus 42.5% for the non-fine-tuned counterpart, while GPT-4o reaches 91.3%. The authors release the dataset and the best model.

Significance. If the reported accuracy is trustworthy, the paper would provide a valuable open-source artifact: a fine-tuned LLM for RTL vulnerability detection and a community database, with training and inference costs far below proprietary alternatives. The release of the dataset, model, and detailed LoRA hyperparameter configuration are explicit strengths. However, the central quantitative claim currently rests on GPT-generated ground truth with no independent validation, no error bars, and no per-class breakdown, so the significance is conditional on substantial additional evaluation work.

major comments (5)
  1. [II.A / III.A] The 4,000 replicated designs are generated by GPT-4/GPT-4o using only temperature and top-p variation, and the paper reports no simulation, formal property checking, or manual audit to confirm that the intended CWE vulnerability was preserved in each replica and that no new vulnerabilities were introduced. If a nontrivial fraction of the labels is wrong, the absolute accuracies and the claimed 40-plus-point improvement over non-fine-tuned models are unsupported. The authors should validate a statistically meaningful sample, for example by running the Cad4Security assertions or known-answer tests on each replica, and report label precision per CWE class.
  2. [II.B / III.C] The fine-tuning responses are generated by GPT-4o, and GPT-4o is also the top proprietary baseline. The reported accuracy may therefore reflect the model learning stylistic fingerprints of GPT-generated Verilog or the wording of GPT-4o rationales rather than genuine vulnerability detection. The authors should break the circularity by evaluating on independently validated labels and by reporting performance separately on hand-written versus GPT-generated test designs, or by ablating with corrupted labels to show the model is not memorizing annotation style.
  3. [III.C / Figure 2] Only a single accuracy number is reported per model, with no confidence intervals, no number of test designs, and no significance test. The headline 40-point gain could fall within the noise if the test set is small or imbalanced. The authors should report per-vulnerability precision/recall/F1, the test set size, and bootstrapped confidence intervals, and they should state how the proprietary baselines were prompted (exact prompt, sampling temperature, number of runs, and whether they used the same test split).
  4. [II.B] The train/test split is described only as 'based on individual designs,' which is ambiguous given that all 13 vulnerabilities originate from 13 base modules and the dataset consists of GPT-generated replicas of those modules. If replicas of the same base design appear in both training and test splits, code-style leakage could inflate accuracy. The authors should clarify whether the split is at the base-module/CWE level and, if not, rerun the evaluation with a strict split that prevents any replica of a given Ariane module from appearing in both training and testing.
  5. [II.B] The paper states that 'a major challenge in dataset creation is the lack of sufficiently detailed annotations' and then uses GPT-4o to create the annotations. This is an admitted limitation that is never addressed in the evaluation: there is no human audit of the GPT-4o-written rationales, nor any measurement of annotation agreement. The authors should at least report a human-evaluation subset with inter-annotator agreement and show how the model's accuracy changes when only validated rationales are used for training.
minor comments (5)
  1. [Abstract / I] The paper should be self-contained regarding the 13 CWE classes; the repeated sentence 'The detailed description of the vulnerabilities will be found here [16]' appears twice in Section II.B and should be replaced with a table summarizing each vulnerability and its base module.
  2. [III.A / III.B] The text says 4,000 designs were generated and then that '4,000 Verilog codes are used during training,' but no split sizes are given. Please state how many designs are in training, validation, and testing, and how the 4000 are distributed across the 13 CWE classes.
  3. [II.B] There are minor capitalization and typographical inconsistencies, including 'LLama' versus 'Llama,' 'Codellama' versus 'CodeLlama,' 'For training, We implemented' with an uppercase W, and 'However, Despite' with a capital D mid-sentence.
  4. [II.B] The list of vulnerabilities contains two entries labeled CWE-310 with different textual descriptions (Trojan in AES for information leakage and Trojan in AES for denial of service), which is confusing; use distinct sub-identifiers or CWE extensions if they exist.
  5. [III.C / Figure 2] Figure 2 is referenced but no figure body is included in the submitted text; the actual figure should include axis labels, per-model error bars, and the number of test samples, and it should be legible in grayscale.

Circularity Check

2 steps flagged · score 6.0 of 10

Evaluation is circular because both the training labels and test labels are produced by GPT-4/GPT-4o, so the reported accuracy measures agreement with the LLM that generated the data, not independent detection of hardware vulnerabilities.

  1. fitted input called prediction [Section II.B (fine-tuning data creation) and Section III.C (result analysis)]
    "A major challenge in dataset creation is the lack of sufficiently detailed annotations. To mitigate this limitation, GPT-4o is employed to generate thorough explanatory annotations across different hardware modules. ... For a rigorous evaluation of the model's effectiveness, the dataset is partitioned into distinct training, validation, and testing subsets based on individual designs."

    The fine-tuned model is trained on prompt-response pairs whose response text is written by GPT-4o, and the test labels are produced by the same GPT-4o annotation pipeline. The reported accuracy (e.g., Mistral-7B at 84.8%) is therefore agreement with GPT-4o's judgments on GPT-generated designs, not agreement with independently verified ground truth. The model is fitted to GPT-4o's output distribution and then 'predicts' test outputs drawn from the same distribution, so the evaluation does not establish detection of actual hardware vulnerabilities.

  2. self definitional [Section II.A (database generation) and Section III.A (dataset replication)]
    "During replication, the specification file corresponding to each vulnerable IP is provided as context to the Replicator LLM, thereby ensuring fidelity to the original functionality and the embedded vulnerability. ... Using this process, a set of 4000 vulnerable SoC hardware designs were generated."

    The 'vulnerable' status of the 4,000 replicated designs is asserted from the LLM replication prompt and the specification file, with no reported simulation, formal property checking, or manual audit confirming that the injected CWE vulnerability survived replication. The ground-truth label is thus defined by the generation process itself: a design is called vulnerable because the Replicator LLM was told to make it vulnerable. Accuracy computed against these self-referential labels cannot independently validate the detector.

full rationale

The paper's strongest claim is that fine-tuning an open-source LLM improves vulnerability detection accuracy by over 40 percentage points (Mistral-7B from 42.5% to 84.8%). Under the paper's own description, however, the entire evaluation loop is closed inside the GPT model family: GPT-4/GPT-4o replicate 4,000 'vulnerable' designs from the authors' Cad4Security specifications (Section II.A, III.A), GPT-4o writes the explanatory annotations that form the fine-tuning targets (Section II.B), and the test split is carved from the same GPT-generated dataset (Section II.B). No independent oracle—simulation, formal verification, or manual audit—is reported to confirm either that the replicated bugs exist or that the GPT-4o annotations are correct. Consequently, the accuracy numbers quantify how well the fine-tuned model imitates GPT-4o's labeling behavior on GPT-generated code, not how well it detects real hardware vulnerabilities. The comparison against GPT-4o as a proprietary baseline is also affected, because GPT-4o is simultaneously the teacher that produced the training signal and the benchmark oracle. This is not a formal derivation circularity, but it is a load-bearing data-provenance circularity: the central empirical claim is unvalidated by any source independent of the model family being evaluated. The paper does acknowledge limitations in general terms, but it does not disclose or mitigate the lack of independent ground-truth validation. Score 6 reflects that the headline metric, as reported, is largely self-referential, while the fine-tuning method itself is not logically impossible and the authors do provide a public dataset and model for external scrutiny.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central evaluation depends on a set of manually chosen training hyperparameters and, more importantly, on the correctness of GPT-generated labels and the soundness of the data split. No new physical or conceptual entities are introduced; the 'BugWhisperer' model and database are artifacts, not postulates.

free parameters (6)
  • LoRA rank = 128
    Chosen manually; controls the number of trainable parameters and thus model capacity, directly affecting the reported accuracy.
  • LoRA alpha = 256
    Chosen manually; scales the LoRA updates and interacts with rank to determine effective learning strength.
  • Dropout = 0.1
    Chosen manually; regularization strength during fine-tuning.
  • Learning rate = 2e-6
    Chosen manually; a low rate intended to preserve pretrained weights.
  • Replicator temperature = 0.6-1.5
    Manually set range for GPT replication calls to balance diversity and functional fidelity of generated designs.
  • Max sequence length = 512
    Chosen manually; truncates context and could affect detection for larger modules.
assumptions (4)
  • domain assumption Cad4Security benchmarks are representative of real SoC vulnerabilities
    Invoked in Section II.A as the source of the 13 vulnerability classes; if these benchmarks differ from real-world designs, the model's utility is narrower.
  • domain assumption GPT-4o-generated annotations correctly indicate whether a specified vulnerability is present
    Used in Section II.B to build prompt-response pairs; no manual or formal validation of these labels is reported.
  • domain assumption The LLM-replicated designs preserve the original vulnerability and intended functionality
    Assumed in Sections II.A and III.A; the paper does not simulate or formally check the generated Verilog.
  • domain assumption The train/test split by 'individual designs' prevents leakage between generated replicas
    Stated in Section II.B; the exact grouping criterion is unspecified, so test-set generalization is not established.

how reviews work

0 comments
Cite this review

Pith. "Pith review of BugWhisperer: Fine-Tuning LLMs for SoC Hardware Vulnerability Detection." pith.science (2026). https://pith.science/paper/I2WYFVBM

@misc{pith2026250522878,
  author       = {Pith},
  title        = {Pith review of: BugWhisperer: Fine-Tuning LLMs for SoC Hardware Vulnerability Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/I2WYFVBM}},
  note         = {Machine review of arXiv:2505.22878}
}
read the original abstract

The current landscape of system-on-chips (SoCs) security verification faces challenges due to manual, labor-intensive, and inflexible methodologies. These issues limit the scalability and effectiveness of security protocols, making bug detection at the Register-Transfer Level (RTL) difficult. This paper proposes a new framework named BugWhisperer that utilizes a specialized, fine-tuned Large Language Model (LLM) to address these challenges. By enhancing the LLM's hardware security knowledge and leveraging its capabilities for text inference and knowledge transfer, this approach automates and improves the adaptability and reusability of the verification process. We introduce an open-source, fine-tuned LLM specifically designed for detecting security vulnerabilities in SoC designs. Our findings demonstrate that this tailored LLM effectively enhances the efficiency and flexibility of the security verification process. Additionally, we introduce a comprehensive hardware vulnerability database that supports this work and will further assist the research community in enhancing the security verification process.

Figures

Figures reproduced from arXiv: 2505.22878 by the authors.

Figure 1
Figure 1. Overview of the proposed BugWhisperer framework. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Performance evaluation of the large proprietary models, fine-tuned and non-fine-tuned open-source models. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

18 extracted references · 8 canonical work pages

  1. [1]

    Security vulnerability analysis of design-for-test exploits for asset protection in socs,

    G. K. Contreras, A. Nahiyan, S. Bhunia, D. Forte, and M. Tehranipoor, “Security vulnerability analysis of design-for-test exploits for asset protection in socs,” in2017 22nd Asia and South Pacific Design Automation Conference (ASP-DAC). IEEE, 2017, pp. 617–622

  2. [2]

    Security and trust vulner- abilities in third-party ips,

    P. Mishra, M. Tehranipoor, and S. Bhunia, “Security and trust vulner- abilities in third-party ips,”Hardware IP Security and Trust, pp. 3–14, 2017

  3. [3]

    Soc secu- rity verification using property checking,

    N. Farzana, F. Rahman, M. Tehranipoor, and F. Farahmandi, “Soc secu- rity verification using property checking,” in2019 IEEE International Test Conference (ITC). IEEE, 2019, pp. 1–10

  4. [4]

    Register transfer level information flow tracking for provably secure hardware design,

    A. Ardeshiricham, W. Hu, J. Marxen, and R. Kastner, “Register transfer level information flow tracking for provably secure hardware design,” in Design, Automation & Test in Europe Conference & Exhibition (DATE),

  5. [5]

    Fuzz, penetration, and ai testing for soc security verification: Challenges and solutions,

    K. Z. Azar, M. M. Hossain, A. Vafaei, H. Al Shaikh, N. N. Mondol, F. Rahman, M. Tehranipoor, and F. Farahmandi, “Fuzz, penetration, and ai testing for soc security verification: Challenges and solutions,” Cryptology ePrint Archive, 2022

  6. [6]

    Arc-fsm-g: Automatic security rule checking for finite state machine at the netlist abstraction,

    R. Kibria, F. Farahmandi, and M. Tehranipoor, “Arc-fsm-g: Automatic security rule checking for finite state machine at the netlist abstraction,” Cryptology ePrint Archive, 2023

  7. [7]

    Socurellm: An llm-driven approach for large-scale system-on-chip security verification and policy generation,

    S. Tarek, D. Saha, S. K. Saha, M. Tehranipoor, and F. Farahmandi, “Socurellm: An llm-driven approach for large-scale system-on-chip security verification and policy generation,”Cryptology ePrint Archive, 2024

  8. [8]

    Llm for soc security: A paradigm shift,

    D. Saha, S. Tarek, K. Yahyaei, S. K. Saha, J. Zhou, M. Tehranipoor, and F. Farahmandi, “Llm for soc security: A paradigm shift,”IEEE Access, vol. 12, pp. 155 498–155 521, 2024

Show all 18 references
  1. [9]

    ChipGPT: How far are we from natural language hardware design,

    K. Chang, Y . Wang, H. Ren, M. Wang, S. Liang, Y . Han, H. Li, and X. Li, “ChipGPT: How far are we from natural language hardware design,” May 2023

  2. [10]

    Llm-assisted generation of hardware assertions,

    R. Kande, H. Pearce, B. Tan, B. Dolan-Gavitt, S. Thakur, R. Karri, and J. Rajendran, “Llm-assisted generation of hardware assertions,”arXiv preprint arXiv:2306.14027, 2023

  3. [11]

    Self-hwdebug: Automation of llm self- instructing for hardware security verification,

    M. Akyash and H. M. Kamali, “Self-hwdebug: Automation of llm self- instructing for hardware security verification,” in2024 IEEE Computer Society Annual Symposium on VLSI (ISVLSI), 2024, pp. 391–396

  4. [12]

    Empowering hardware security with llm: The development of a vul- nerable hardware database,

    D. Saha, K. Yahyaei, S. K. Saha, M. Tehranipoor, and F. Farahmandi, “Empowering hardware security with llm: The development of a vul- nerable hardware database,” in2024 IEEE International Symposium on Hardware Oriented Security and Trust (HOST). IEEE, 2024, pp. 233– 243

  5. [13]

    Gpt-4 technical report

    “Gpt-4 technical report.” [Online]. Available: https://arxiv.org/pdf/2303. 08774.pdf

  6. [14]

    The llama 3 herd of models,

    A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Yang, A. Fanet al., “The llama 3 herd of models,”arXiv preprint arXiv:2407.21783, 2024

  7. [15]

    Code llama: Open foundation models for code,

    B. Roziere, J. Gehring, F. Gloeckle, S. Sootla, I. Gat, X. E. Tan, Y . Adi, J. Liu, R. Sauvestre, T. Remezet al., “Code llama: Open foundation models for code,”arXiv preprint arXiv:2308.12950, 2023

  8. [16]

    Bench- marking of soc-level hardware vulnerabilities: A complete walkthrough,

    S. Tarek, H. A. Shaikh, S. R. Rajendran, and F. Farahmandi, “Bench- marking of soc-level hardware vulnerabilities: A complete walkthrough,” in2023 IEEE Computer Society Annual Symposium on VLSI (ISVLSI), 2023, pp. 1–6

  9. [17]

    Lora: Low-rank adaptation of large language models,

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, and W. Chen, “Lora: Low-rank adaptation of large language models,” arXiv preprint arXiv:2106.09685, 2021

  10. [2017]

    1691–1696

    IEEE, 2017, pp. 1691–1696

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.