Leveraging SystemC-TLM-based Virtual Prototypes for Embedded Software Fuzzing

Chiara Ghinami; Jonas Winzer; Lennart M. Reimann; Lukas J\"unger; Nils Bosbach; Rainer Leupers; Simon W\"orner

REVIEW 4 major objections 6 minor 18 references

Leveraging SystemC-TLM-based Virtual Prototypes for Embedded Software Fuzzing

T0 review · 4 major / 6 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read SystemC virtual prototypes can serve as peripheral-aware fuzzing engines at QEMU-AFL speed.

desk verdict A clean, useful integration of AFL++ with SystemC-TLM virtual prototypes, but the evaluation is too thin to back the headline performance claim. read the letter →

arxiv 2509.01318 v1 pith:FTLHQWRW submitted 2025-09-01 cs.SE

Chiara Ghinami , Jonas Winzer , Nils Bosbach , Lennart M. Reimann , Lukas Jünger , Simon Wörner , Rainer Leupers This is my paper

classification cs.SE

keywords virtualprototypesSystemCTLMfuzzingAFL++MMIOmodelingembeddedfirmwaregreybox

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that SystemC-based virtual prototypes, normally used for pre-silicon software testing, can be turned into fuzzing engines for embedded firmware by cleanly decoupling the fuzzer from the simulator. The authors build a harness that lets an AFL-based fuzzer drive a SystemC simulator without modifying either side, and a plug-in that intercepts peripheral reads and answers them with fuzzer-supplied values. They demonstrate this on a UART password-checking bare-metal program and a Zephyr OS version, running on two different SystemC simulators. Their central performance claim is that persistent mode reaches thousands of executions per second, comparable to QEMU-AFL, while also modeling peripherals that QEMU user-mode cannot.

What carries the argument

The core mechanism is the pairing of (1) a generic harness interface between AFL++ and the SystemC virtual prototype that manages VP process lifecycle and forwards coverage and crash signals, and (2) the MMIO tracking probe, a TLM-level plug-in inserted between CPU and bus that intercepts memory-mapped read transactions and returns fuzzer-generated data. A persistent mode jumps the simulated program back to its entry point instead of restarting the whole VP process, removing the dominant startup cost and making per-execution speed comparable to QEMU-AFL.

What would settle it

Run the same fuzzing campaign on real nRF51 hardware executing the same UART password program and check whether the crashes discovered in the SystemC virtual prototype reproduce; if VP-only crashes do not reproduce, or if real-hardware bugs are missed because peripheral models are absent, the transfer assumption fails.

Watch

Extended reading notes

Core claim

The paper's central claim is that SystemC-TLM virtual prototypes can be turned into fuzzing targets without reimplementing fuzzing logic inside the simulator. The authors' contribution is a decoupling harness that connects AFL++ to any SystemC virtual prototype, and an MMIO tracking probe that sits between CPU and bus, intercepts reads to a user-defined peripheral address range, and substitutes fuzzer-generated values for the peripheral's reply. On a Cortex-M0 nRF51 model running a UART password-checking program, both bare-metal and under Zephyr OS, they report thousands of executions per second in persistent mode, which they say is comparable to QEMU-AFL while additionally simulating periph

Load-bearing premise

The whole approach assumes the SystemC virtual prototype and its peripheral models represent the target hardware faithfully enough that crashes and coverage found in simulation transfer to the real device; the paper's own evaluation excludes most peripherals (GPIO, SPI, I2C, CAN) from the simulated environment.

Editorial extensions

If this is right

Firmware fuzzing can run in the same SystemC-VP environment used for pre-silicon development, so bugs found during fuzzing relate directly to the models a team already maintains.
Peripheral-aware fuzzing becomes possible without writing custom QEMU device emulation: specifying an address range in a config file is enough to start fuzzing a peripheral's reads.
OS-level firmware, including full boot and peripheral initialization, can be fuzzed as a whole—something QEMU user-mode fuzzing cannot do.
The persistent mode keeps VP-based fuzzing throughput within striking distance of QEMU-AFL, so the added peripheral support does not come with an order-of-magnitude performance penalty.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial inference: the same decoupling pattern could extend to other coverage-guided fuzzers beyond AFL++, since the harness presents a generic feedback loop rather than AFL-specific glue.
Editorial inference: the probe treats all tracked peripheral reads as fuzzer-controlled, but many real peripherals impose protocol order (e.g., status flags before data); a natural extension is to enforce read-ordering semantics inside a lightweight peripheral model, which the current single-UART demonstration does not exercise.
Editorial inference: because the probe is an address-range filter, tracking multiple peripheral ranges simultaneously could enable fuzzing of multi-peripheral interaction bugs, a scenario the paper's single-tracked-peripheral evaluation does not cover.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The genuinely new piece here is the decoupling harness between AFL++ and SystemC-TLM virtual prototypes, plus the MMIO probe that intercepts peripheral reads and feeds fuzzer-generated values, and the persistent mode that avoids restarting the VP per run. That combination is not in the cited literature, which is mostly QEMU-based rehosting. The engineering looks solid: they demonstrate two different simulators (AVP32 and SIM-A), the persistent mode clearly helps, and the numbers are internally consistent. Credit is also due for linking the code and for engaging honestly with Fuzzware and related work.

The soft spots are real, but not fatal. The evaluation is a single synthetic UART password check and a Zephyr port of the same program. There are no error bars, no coverage or bug-discovery metrics, and no direct comparison to Fuzzware, which is the closest prior art. The performance claim is specifically about executions per second, so the stress test's complaint that they don't show bug finding is fair but slightly against what they claimed: they only said 'comparable performance' in throughput. Still, the broader implication that this enables useful peripheral-aware fuzzing is unproven. The MMIO substitution simply replaces every tracked read with the fuzzer's next bytes, with no consistency checks on FIFO flags, readiness, or timing. That can create impossible hardware states, and with most peripherals excluded, the Zephyr test is not strong evidence about real nRF51 behavior. Also, the one-size-fits-all claim for 'any SystemC-based VP' rests on testing two, both from the same group.

Minor notes: the code is not commit-pinned, SIM-A is proprietary, and the comparison indirectly promotes tools from the same research group and a company where one author works. That doesn't make the work wrong, but it invites a skeptical look at the performance numbers. None of this is disqualifying. The paper is a well-scoped workshop-style contribution, and the core idea is plausible and reproducible enough to deserve a serious referee. My recommendation: send it to peer review, but require the authors to either soften the generality claims or add one real bug-finding case and check whether the MMIO value substitution preserves enough hardware semantics to guide the fuzzer.

For the right audience—people building virtual prototypes and doing embedded fuzzing—this is a useful data point, even if it doesn't resolve the fidelity question.

Referee Report

4 major / 6 minor

Summary. The paper presents a framework for fuzzing embedded software on SystemC-TLM virtual prototypes with AFL++. It contributes (i) a probe plug-in that intercepts MMIO read transactions and substitutes fuzzer-generated values, (ii) a harness that decouples the AFL++ fuzzer from VP process management, and (iii) a persistent mode that avoids restarting the VP for each test case. The authors integrate the framework with two Cortex-M0 simulators (AVP32 and SIM-A) and evaluate it on a bare-metal password-checking program and the same program under Zephyr OS. They report restart vs. persistent throughput and compare persistent-mode executions/s with QEMU-AFL, claiming comparable performance while integrating additional peripheral support.

Significance. If the central claim is substantiated, the work provides a reusable way to combine AFL++ with SystemC VPs, which are widely used in embedded software development. The open-source release, the modular harness, and the persistent-mode mechanism are concrete practical strengths. However, the evaluation is currently too thin to establish the claimed peripheral support and the fuzzing effectiveness: it measures raw throughput on one synthetic example, with no coverage, crash-discovery, or model-fidelity evidence.

major comments (4)

[Section 4.2 (persistent mode)] The persistent mode 'jumps to the entry point' on exit, but the paper does not state what state is reset: CPU registers, RAM contents, peripheral and MMIO probe state, coverage bitmap, or OS state. If state is not reset, consecutive inputs are not independent, which can lead to cross-input contamination, false crashes, or missed crashes. Since all persistent-mode throughput numbers in Figures 3 and 4 rely on this mechanism, please specify the reset behavior or justify that the test workload is insensitive to residual state.
[Section 4.1 (MMIO tracking)] The probe returns a fuzzer-generated value on every tracked read without modeling device semantics or timing. For example, a UART status register can report data available while the RX FIFO is empty, and repeated reads can return different values without any corresponding write. This can create impossible hardware states and may mislead coverage-guided exploration. Since this mechanism is the basis for 'additional peripheral support,' the paper needs evidence that the substituted reads preserve realistic behavior, e.g., a comparison against real hardware, a validated peripheral model, or an explicit consistency invariant enforced by the probe. No such validation is currently reported.
[Section 5 (Figures 3-4, Table 1)] The performance comparison is based on execution/s for a single synthetic password-checking program. No repetitions, standard deviations, or confidence intervals are provided, so it is impossible to assess whether the reported gap to QEMU-AFL is significant. Moreover, throughput alone is not a measure of fuzzing effectiveness; the paper does not report coverage, path counts, or bugs found on any nontrivial target. To support the central claim, the evaluation needs repeated measurements and a more realistic workload with coverage or crash-discovery metrics.
[Section 5 (peripheral coverage)] The conclusion claims the framework 'integrates additional peripheral support,' but the experiments track a single UART and include only one timer; the text explicitly excludes GPIOs, SPI, I2C, CAN, and most timers. The reported demonstration does not substantiate broad peripheral support. Please either limit the claim to the demonstrated peripherals or extend the evaluation to include additional peripheral interactions and show that the fuzzer can meaningfully drive them.

minor comments (6)

[Section 2.1 / Section 6] The simulator library is called 'Fast Translation Library (FTL)' in the background section but 'Fast Transition Level Library' in the conclusion. Use one consistent name.
[Table 1] The heading 'Execution speed (in milliseconds)' is misleading; the values are elapsed times, not speeds. Rename to 'Execution time (ms)' or similar.
[Listing 1] The constant 'encr_password' is never defined. Specify the actual password value or a placeholder so the experiment is reproducible.
[Figure 4] The caption lists QEMU alongside AVP32 and SIM-A, but the text states QEMU cannot execute the Zephyr workload. The Zephyr group in the figure appears to have only two bars. Clarify the figure or the caption.
[Abstract / Introduction] The phrase 'various softwares' is grammatically awkward; use 'software programs' or 'embedded applications'.
[Section 3] The related work section mentions Fuzzware's MMIO tracking but does not clearly differentiate the proposed probe from it. Since the mechanism is conceptually similar, a sentence explaining what the SystemC/VCML probe adds beyond Fuzzware's model would help position the contribution.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the MMIO substitution is the designed mechanism, not a derived result, and the performance claim is an empirical comparison against an external baseline.

full rationale

The paper's contribution is an integration mechanism plus an empirical benchmark, not a derivation from assumptions. Section 4.1 describes the probe as intercepting peripheral reads and returning fuzzer-generated values; this is the intended runtime behavior, and the paper makes no claim that is logically forced by that mechanism. The 'comparable performance to QEMU-AFL' claim (Section 6) is supported by measured execs/sec against QEMU-AFL in Figure 4, an external baseline, not by a fitted parameter or by the authors' own prior results. No parameter is fitted to the target program; the only configuration choices are user-specified MMIO address ranges and breakpoints (Section 4.2). References to AVP, SIM-A, and VCML are tool attribution and background; although one author is affiliated with MachineWare, these citations do not carry the load of any inference about correctness or effectiveness. The paper's own stated limitations (excluded peripherals, the synthetic password example) are threats to external validity, not circular steps. No equation or claim reduces to its own input by construction, so the circularity score is 0.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

This engineering paper does not build a mathematical derivation; the ledger records the modeling and tooling assumptions on which the fuzzing results rest. No free parameters are fitted: the MMIO address range, persistent-mode entry and exit addresses, and Caesar shift are user-selected test configurations, not quantities inferred from data.

assumptions (5)

domain assumption SystemC/TLM transaction semantics, as implemented by AVP32, SIM-A and VCML, correctly model the Cortex-M0/nRF51 platform.
The framework's claim to test real embedded software depends on the VP accurately representing target hardware; introduced in Section 2.1 and used throughout the evaluation.
domain assumption The error oracle (R0==1 for bare-metal main return, breakpoint on error handler for OS systems) reliably identifies program failure.
Section 4.2 defines crash and error detection through these mechanisms; if a program fails in another way, the fuzzer may miss it.
domain assumption Coverage instrumentation in the simulators gives AFL++ correct and sufficiently fine-grained feedback for greybox guidance.
Persistent mode and coverage collection in Section 4.2 assume the VP reports coverage equivalent to real execution paths.
domain assumption Intercepting an MMIO read and substituting a fuzzer-supplied value does not desynchronize peripheral or software state.
The probe in Section 4.1 returns values for tracked reads while untracked peripherals (GPIO, SPI, I2C, CAN, most timers) are omitted; state effects of omitted peripherals are not modeled.
domain assumption Jumping to the entry address in persistent mode restores sufficient program state for each new input to be tested independently.
Section 4.2 says the VP jumps to the entry point on exit; if global state persists incorrectly, coverage and error signals can be contaminated across inputs.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Leveraging SystemC-TLM-based Virtual Prototypes for Embedded Software Fuzzing." pith.science (2026). https://pith.science/paper/FTLHQWRW

@misc{pith2026250901318,
  author       = {Pith},
  title        = {Pith review of: Leveraging SystemC-TLM-based Virtual Prototypes for Embedded Software Fuzzing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FTLHQWRW}},
  note         = {Machine review of arXiv:2509.01318}
}

read the original abstract

SystemC-based virtual prototypes have emerged as widely adopted tools to test software ahead of hardware availability, reducing the time-to-market and improving software reliability. Recently, fuzzing has become a popular method for automated software testing due to its ability to quickly identify corner-case errors. However, its application to embedded software is still limited. Simulator tools can help bridge this gap by providing a more powerful and controlled execution environment for testing. Existing solutions, however, often tightly couple fuzzers with built-in simulators that lack support for hardware peripherals and offer limited flexibility, restricting their ability to test embedded software. To address these limitations, we present a framework that allows the integration of American-Fuzzy-Lop-based fuzzers and SystemC-based simulators. The framework provides a harness to decouple the adopted fuzzer and simulator. In addition, it intercepts peripheral accesses and queries the fuzzer for values, effectively linking peripheral behavior to the fuzzer. This solution enables flexible interchangeability of peripherals within the simulation environment and supports the interfacing of different SystemC-based virtual prototypes. The flexibility of the proposed solution is demonstrated by integrating the harness with different simulators and by testing various softwares.

Figures

Figures reproduced from arXiv: 2509.01318 by the authors.

**Figure 1.** The fuzz testing loop. 2.2 Fuzz Testing Fuzzing is a widely used technique for assessing software correctness and reliability by automatically generating and executing test inputs. Greybox fuzzers like AFL leverage coverage feedback to guide input mutation and explore new execution paths ( [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗

**Figure 2.** The VP with the probe component. 4.1 MMIO Tracking Plug-in Whenever the program performs a read operation within the peripheral’s address range, the tracking system intercepts the request and retrieves the value generated by the fuzzer ( [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** The simulator’s performance for different execution modes. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Figures from the paper (1 more)

**Figure 4.** Figure 4: Comparison of AVP32, SIM-A and QEMU executing in persistent mode. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

18 extracted references · 18 canonical work pages

[1]

Qemu, a fast and portable dynamic translator

Fabrice Bellard. Qemu, a fast and portable dynamic translator. InProceedings of the Annual Conference on USENIX Annual Technical Conference , 2005. 8 C. Ghinami et al

work page 2005
[2]

Towards automated dynamic analysis for linux-based em- bedded firmware

Daming D Chen et al. Towards automated dynamic analysis for linux-based em- bedded firmware. InNDSS, 2016

work page 2016
[3]

Sok: Enabling security analyses of embedded systems via rehosting

Andrew Fasano et al. Sok: Enabling security analyses of embedded systems via rehosting. In Proceedings of the 2021 ACM Asia conference on computer and communications security, 2021

work page 2021
[4]

{P2IM}: Scalable and hardware-independent firmware testing via automatic peripheral interface modeling

Bo Feng et al. {P2IM}: Scalable and hardware-independent firmware testing via automatic peripheral interface modeling. InUSENIX Security 20 , 2020

work page 2020
[5]

In 14th USENIX WOOT 20 , 2020

Andrea Fioraldi et al.{AFL++}: Combining incremental steps of fuzzing research. In 14th USENIX WOOT 20 , 2020

work page 2020
[6]

Libafl: A framework to build modular and reusable fuzzers

Andrea Fioraldi et al. Libafl: A framework to build modular and reusable fuzzers. In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Commu- nications Security, 2022

work page 2022
[7]

Dissecting american fuzzy lop: a fuzzbench evaluation.ACM transactions on software engineering and methodology , 2023

Andrea Fioraldi et al. Dissecting american fuzzy lop: a fuzzbench evaluation.ACM transactions on software engineering and methodology , 2023

work page 2023
[8]

Fast SystemC Processor Models with Unicorn

Jünger et al. Fast SystemC Processor Models with Unicorn. InProceedings of the Rapid Simulation and Performance Evaluation: Methods and Tools . Association for Computing Machinery, 2019

work page 2019

Show all 18 references

[9]

MachineWare. VCML. URL: https://github.com/machineware-gmbh/vcml
[10]

SIM-V DVCON Proceedings, 2022

MachineWare. SIM-V DVCON Proceedings, 2022. URL: https:// dvcon-proceedings.org/wp-content/uploads/74137.pdf

2022
[11]

Machineware website, 2025

MachineWare. Machineware website, 2025. URL:https://www.machineware.de/

2025
[12]

Fuzzware: Using precise{MMIO} modeling for effective firmware fuzzing

Tobias Scharnowski et al. Fuzzware: Using precise{MMIO} modeling for effective firmware fuzzing. InUSENIX Security 22 , 2022

2022
[13]

nrf51 soc

Nordic Semiconductor. nrf51 soc. URL:https://www.nordicsemi.com/Products/ nRF51822
[14]

Systemc website, 2025

SystemC. Systemc website, 2025. URL:https://systemc.org/

2025
[15]

Fuzzing of embedded systems: A survey.ACM Computing Surveys, 2022

Joobeom Yun et al. Fuzzing of embedded systems: A survey.ACM Computing Surveys, 2022

2022
[16]

American fuzzy lop-whitepaper.Retrieved September, 2016

Michał Zalewski. American fuzzy lop-whitepaper.Retrieved September, 2016

2016
[17]

Zephyr OS Website

Zephyr. Zephyr OS Website. URL:https://www.zephyrproject.org/
[18]

Firm-afl: high-throughput greybox fuzzing of iot firmware via augmented process emulation

Yaowen Zheng et al. Firm-afl: high-throughput greybox fuzzing of iot firmware via augmented process emulation. InProceedings of the 28th USENIX Conference on Security Symposium, 2019

2019

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.

[1] [1]

Qemu, a fast and portable dynamic translator

Fabrice Bellard. Qemu, a fast and portable dynamic translator. InProceedings of the Annual Conference on USENIX Annual Technical Conference , 2005. 8 C. Ghinami et al

work page 2005

[2] [2]

Towards automated dynamic analysis for linux-based em- bedded firmware

Daming D Chen et al. Towards automated dynamic analysis for linux-based em- bedded firmware. InNDSS, 2016

work page 2016

[3] [3]

Sok: Enabling security analyses of embedded systems via rehosting

Andrew Fasano et al. Sok: Enabling security analyses of embedded systems via rehosting. In Proceedings of the 2021 ACM Asia conference on computer and communications security, 2021

work page 2021

[4] [4]

{P2IM}: Scalable and hardware-independent firmware testing via automatic peripheral interface modeling

Bo Feng et al. {P2IM}: Scalable and hardware-independent firmware testing via automatic peripheral interface modeling. InUSENIX Security 20 , 2020

work page 2020

[5] [5]

In 14th USENIX WOOT 20 , 2020

Andrea Fioraldi et al.{AFL++}: Combining incremental steps of fuzzing research. In 14th USENIX WOOT 20 , 2020

work page 2020

[6] [6]

Libafl: A framework to build modular and reusable fuzzers

Andrea Fioraldi et al. Libafl: A framework to build modular and reusable fuzzers. In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Commu- nications Security, 2022

work page 2022

[7] [7]

Dissecting american fuzzy lop: a fuzzbench evaluation.ACM transactions on software engineering and methodology , 2023

Andrea Fioraldi et al. Dissecting american fuzzy lop: a fuzzbench evaluation.ACM transactions on software engineering and methodology , 2023

work page 2023

[8] [8]

Fast SystemC Processor Models with Unicorn

Jünger et al. Fast SystemC Processor Models with Unicorn. InProceedings of the Rapid Simulation and Performance Evaluation: Methods and Tools . Association for Computing Machinery, 2019

work page 2019