Pith. sign in

REVIEW 4 major objections 5 minor 40 references

FirmPilot claims that an evidence-grounded multi-agent loop raises web-service reachability on a 10,033-image IoT corpus from 25.49% to 52.39%.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 00:43 UTC pith:6Y5I3OLH

load-bearing objection FirmPilot is a promising, well-specified agentic rehosting system whose headline gains are plausible but rest on unreleased artifacts and a single run; the multi-agent design is the real novelty. the 4 major comments →

arxiv 2607.14903 v1 pith:6Y5I3OLH submitted 2026-07-16 cs.SE

FirmPilot: Evidence-Guided Multi-Agent Environment Recovery for IoT Firmware Rehosting

Yanbing Shen , Fan Zhang , Haitao Xu This is my paper
classification cs.SE
keywords firmware rehostingIoT securitymulti-agent LLMNVRAM state recoverynetwork exposure recoveryemulationLFwC corpusexecution-grounded validation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Firmware rehosting — running IoT device firmware in an emulator so security analysts can probe it — usually breaks on real-world images because boot scripts, persistent settings, and network configuration are tightly coupled. This paper argues that the right repair is iterative environment reconstruction: let a small set of specialized agents propose small, typed changes to the boot, state, and network layers, and accept a change only if a fresh emulated run and fixed probes confirm it. On 10,033 downloadable images from the LFwC corpus, the resulting system lifts web-service reachability from 25.49% to 52.39%, and network reachability from 39.30% to 71.93%. A general-purpose coding agent reaches only 5.43% on the same corpus, and the recovered services support downstream use such as RouterSploit checks and protocol-aware fuzzing. If the results hold, the practical consequence is that far more of the long tail of vendor-specific firmware becomes analyzable without per-device hand-tuning.

Core claim

The central claim is that firmware-rehosting failure is not primarily a missing-tool problem but an environment-reconstruction problem, and that it can be solved by an evidence-indexed loop. Each iteration runs the current artifact state in QEMU, collects serial logs and probe results, retrieves firmware-specific evidence, plans an ordered set of recovery actions, applies the chosen deltas through typed interfaces (filesystem/init, NVRAM key–value overlay, network exposure), and re-runs fixed probes before accepting the change. The authors model this as a bounded transition system in which LLM outputs are hypotheses rather than direct edits. On the LFwC corpus the paper reports that Web reac

What carries the argument

The load-bearing object is the bounded, typed action interface and the re-execution acceptance gate. Every proposal from the SEARCH, PLAN, FILE, NVRAM, and NETWORK agents must be expressed as a structured delta — a startup command, a key/value/guard tuple, or an IP/interface/bridge/port tuple — which is rendered by deterministic tooling and validated by the next emulated run plus fixed probes. This turns model reasoning into a constrained control plane, separates inference from mutation, and produces an execution-indexed causal trace (artifact versions, observations, retrieved evidence, accepted deltas) that makes each success attributable to a concrete environment transition.

Load-bearing premise

The gain rests on the 10,033 downloadable images being a fair sample of LFwC and on the baseline runs being given exactly the same 2,400-second budget and probe definitions; if the excluded 880 images or the baseline executions differ systematically, the two-fold reachability gap could be an artifact of the comparison protocol.

What would settle it

A third-party replication on a random 1,000-image subset of LFwC, using the identical per-image 2,400-second budget and the paper's fixed HTTP(S)/ICMP probes, with both the template-driven baseline and FirmPilot running on the same 10,033-image denominator: if the baseline's web reachability approaches 52.39%, or if including the 880 dead-link images by re-downloading from archives changes the reported 52.39% by more than a few points, the central claim collapses.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Across the 10,033 LFwC images, the reported absolute gains are 2,699 additional web-reachable and 3,274 additional network-reachable firmware images over the template-driven baseline.
  • The 5,256 web-reachable rehosts support service discovery (16,290 service records, 57 labels), 408 strict RouterSploit positives (193 manually validated), and protocol-aware fuzzing with crash candidates on 1,565 images — turning reachability into a substrate for downstream security workflows.
  • Ablations show planning and NVRAM state synthesis are the largest contributors; removing the planner loses 2,472 of the 5,256 Web successes, and removing NVRAM recovery loses 2,003.
  • A general-purpose coding agent with the same budget and evidence reaches only 5.43% Web reachability and uses roughly 291 times more tokens per image, indicating the domain-specific control plane, not raw LLM capability, is what carries the gain.
  • On the public 1,122-image FirmAE benchmark the system preserves near-baseline performance and slightly improves Web reachability from 79.4% to 82.2%, so the method does not sacrifice well-covered settings.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same evidence-loop idea could be applied to other partially observable reconstruction tasks in emulation, such as recovering missing kernel modules, TLS certificates, or vendor wrappers, wherever the acceptance signal is a re-run and a fixed probe.
  • A direct test left implicit by the paper: hold the loop fixed and swap the underlying LLM (here DeepSeek V4 Flash) to see how much of the gain is due to the evidence gate versus the model; the ablations keep the model fixed and therefore cannot separate those.
  • The per-image execution records that drive the loop could serve as a reusable benchmark for future rehosting systems, giving the community a common, auditable denominator — but only if the records and code are released.
  • The 880 LFwC images excluded because of dead vendor links may not be missing at random; if those devices are systematically older or rarer, the reported 52.39% could differ under a complete corpus, which a sensitivity analysis would show.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper presents FirmPilot, an evidence-guided multi-agent framework for IoT firmware rehosting. It reformulates rehosting as iterative environment reconstruction in which a search agent retrieves evidence, a planner schedules actions, and file/NVRAM/network agents produce typed, execution-validated deltas. The central empirical claim, evaluated on 10,033 LFwC images, is that FirmPilot raises web-service reachability over FirmAE from 25.49% to 52.39% and network reachability from 39.30% to 71.93%, while also enabling downstream workflows such as RouterSploit interaction and protocol-aware fuzzing. The authors report leave-one-module ablations, a Claude Code baseline, and a public FirmAE benchmark comparison.

Significance. If the reported measurements are accurate, the paper would constitute a substantial advance in automated firmware rehosting: roughly doubling service-facing reachability on a large, realistic corpus is practically important, and the design principle of bounded, execution-accepted transitions is well motivated. The evaluation protocol is unusually explicit about probes, budgets, and per-image records, which is a strength. However, the central claim rests on a single unreleased run against FirmAE, with no code, data, error bars, or reruns; the RQ2 ablation is computed only on the 5,256 full-system successes rather than the full corpus; and the RQ3 baseline differs in both the action space and the underlying LLM. These issues currently make the magnitude of the claimed gain difficult to verify independently.

major comments (4)
  1. [§V-B and Introduction (contribution 3)] The Introduction states 'Full-corpus ablations attribute the gain to planning, state synthesis...' but §V-B explicitly says 'We audit the same 5,256 Web-success images used throughout the downstream workflow analysis' and 'RQ2 measures each role on the 5,256 accepted Web-success executions from the full system.' Ablating only the base successes measures the conditional contribution to maintaining those successes, not the contribution to the full-corpus gain. For example, 'no-Plan loses 2,472 images' means 2,472 of 5,256; it does not say how many of the 4,777 failures would become successes with planning. The full-corpus attribution claim is therefore unsupported. Please either rerun ablations on the full 10,033-image set or rephrase the claims to match the actual ablation basis.
  2. [§V-A and §V-E] The headline improvement (25.49% to 52.39% web; 39.30% to 71.93% network) depends entirely on a single unreleased run of FirmAE and FirmPilot. The 'Baseline scope' paragraph in §V-E asserts that FirmAE used 'the same probes' and a 2,400-second budget, but the manuscript provides no invocation script, no TAP/bridge configuration, no exported guest-IP list, and no per-image records to confirm that the baseline was deployed correctly. FirmAE's LFwC web rate (25.49%) is far below its own public-benchmark rate (79.4% on 1,122 images); this is plausible for long-tail diversity, but it is also exactly what a misconfigured network setup (e.g., missing TAP, wrong probe IPs) would produce. Because the central claim is a 2x gap, this is a load-bearing verifiability issue. Release the code, baseline configuration, and execution records, or provide independent rerun variance.
  3. [§V-C, RQ3] The RQ3 comparison uses Claude Code as the 'general-purpose coding agent,' while all FirmPilot runs use DeepSeek V4 Flash as the LLM backend. The comparison therefore varies both the domain-specific action space and the underlying model. The conclusion that 'a general-purpose coding agent does not recover the same service surface when it lacks the domain action space' is confounded: the lower reachability of Claude Code could be due to the model, the prompting, the tool harness, or any combination. To support the claimed domain-specificity, control the model across conditions (e.g., run the FirmPilot agents on the same Claude model, or run a general coding agent on DeepSeek) or otherwise ablate the model factor.
  4. [§V-D, Table VI and Table VII] RQ4 states 'Manual validation confirms 193 reproducible RouterSploit findings' without providing a validation protocol, selection criteria, or inter-rater agreement. Similarly, fuzzing results are reported as 'crash candidates' deduplicated by log-tail markers; the text generally uses 'crash candidate' carefully, but the RouterSploit 'manual validation' number is presented as a confirmed finding. Please specify who performed the manual validation, what defined reproducibility, and whether the 193 findings were pre-registered or independently checked. Without this, the RQ4 evidence is not auditable beyond the authors' own assertion.
minor comments (5)
  1. [Introduction] The third contribution bullet uses 'Full-corpus ablations' but §V-B's ablations are on the 5,256 web-success subset. Reconcile this wording with the actual methodology.
  2. [Table III] 'A VM' in the vendor list should be 'AVM.' Also check spacing in Table I ('PANDaW AN').
  3. [§V-C] The token comparison between Claude Code (3.61M tokens/image) and FirmPilot (12.4K tokens/image) should clarify whether the counts include system prompts, tool outputs, retries, and 'latest-result aggregation,' and should specify the exact model/version of Claude Code used.
  4. [§V-A] The statement that FirmAE's Docker evaluation uses a 2,400-second per-firmware check is not cited. Add a reference or specify where this workflow is described.
  5. [§IV-B] The formalization uses 'x_t ⊕ ...' and 'L_{a in p_t}' without defining the semantics of ⊕ and sequential composition beyond a prose sentence. A one-line formal definition would improve rigor.

Circularity Check

0 steps flagged

No significant circularity: the central claims are empirical measurements against externally defined probes, with no fitted-parameter predictions or self-citation chains carrying the result.

full rationale

FirmPilot's core claims are measured outcomes on a fixed corpus (10,033 LFwC images) using externally defined predicates: 'Network reachable requires a standardized ICMP response' and 'Web-service reachable requires a completed HTTP(S) transaction to an exported guest-IP candidate.' The reported gains (25.49% to 52.39% web, 39.30% to 71.93% network) are per-image execution records comparing FirmPilot to FirmAE and Claude Code under the same 2,400-second budget and the same probes. No parameter is fitted to the target metric, and no 'prediction' is derived from the evaluation outcome. The system's acceptance loop uses the same reachability probes as its objective, but that is goal-directed behavior rather than a circular derivation: measuring whether an agent reaches a designated state is not equivalent to defining that state in terms of the measurement. The ablation study (RQ2) holds the runnable substrate fixed and removes one agent at a time, yielding causally interpretable comparisons; even if the ablations were imperfect, they do not reduce the main result to its inputs. There are no load-bearing self-citations: references are to prior systems (FirmAE, QEMU, LFwC, LLM-agent frameworks) used as baselines or background, not to the authors' own claims. The absence of released code or execution records is a reproducibility and baseline-fairness concern, not a circularity concern. The paper's own threats-to-validity section acknowledges dataset and baseline-scope limitations, but it does not claim to derive the result from any fitted or self-referential premise. Overall, the derivation chain is self-contained as an empirical evaluation, so the circularity score is 0.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

The central claim rests on an empirical system evaluation, not on a derivation. The main hand-set choices are budgets and probe thresholds; the main domain assumptions are about representativeness of the corpus and validity of the reachability predicate.

free parameters (4)
  • Planner round limit = 3
    At most three planner rounds within 2,400s; chosen by hand, not justified against other budgets (Section V Execution policy).
  • Search snippet admission = top-5
    SEARCH admits only the highest-ranked cited snippets, top-5; hand-set and not ablated (Section IV-C).
  • Probe timeout = 2 seconds
    HTTP(S) request timeout inside image budget; affects the web-reachability predicate (Section V Execution policy).
  • LLM backend = DeepSeek V4 Flash
    All agents use this model; results may not transfer to other LLMs, and model variability is acknowledged but not quantified (Section V).
axioms (4)
  • domain assumption QEMU whole-system emulation preserves enough device behavior that externally reachable services in emulation correspond to real firmware behavior.
    The entire evaluation treats QEMU-based reachability as the success criterion (Sections II-A and V).
  • domain assumption The LFwC executable subset (10,033 images after dropping 880 dead links) is representative of real-world IoT firmware and unbiased against baselines.
    Section V Dataset: dead vendor links removed; no distributional check vs excluded images is reported.
  • domain assumption The web-service reachability predicate (completed HTTP(S) transaction to exported guest IP, parseable management response, 2s timeout; status 200 not required) is a meaningful measure of rehosting utility.
    Section V Execution policy defines success; RQ4 partially validates downstream utility, but the primary metric is this probe.
  • domain assumption Retrieved Web/GitHub evidence is trustworthy and not contaminated by the authors' own prior outputs.
    SEARCH admits cited snippets with provenance; no filtering for self-sourced or generated content is described (Section IV-C).

pith-pipeline@v1.3.0-alltime-deepseek · 16551 in / 12099 out tokens · 104003 ms · 2026-08-02T00:43:44.436444+00:00 · methodology

0 comments
read the original abstract

Firmware rehosting executes firmware images in emulated environments such as QEMU to enable scalable dynamic analysis of Internet of Things (IoT) devices. In practice, rehosting pipelines remain fragile across diverse real-world firmware images, as reaching an externally observable execution state depends on tightly coupled artifacts spanning boot scripts, persistent configuration (e.g., NVRAM-like key-value state), and network setup. Template-driven frameworks often fail to accommodate long-tail vendor conventions, while unconstrained use of large language models (LLMs) risks unsupported modifications and irreproducible executions. We introduce FirmPilot, an evidence-guided multi-agent framework for environment recovery in firmware rehosting. FirmPilot reformulates rehosting as iterative environment reconstruction in which a search agent grounds decisions through similarity-based retrieval, a planner coordinates execution-accepted transitions, and specialized agents recover filesystem/init artifacts, persistent state, and network exposure. Through repeated execution and evidence-grounded artifact deltas, the system resolves cross-layer dependencies across boot, state, and networking that otherwise prevent firmware executions from reaching a stable, externally reachable state in emulation. Evaluated on the large-scale, real-world LFwC firmware corpus, FirmPilot improves web-service reachability over FirmAE from 25.49% to 52.39% and network reachability from 39.30% to 71.93%. The resulting rehosts raise the average number of detected services per firmware from 0.86 to 1.62 and support downstream analysis workflows, including RouterSploit interaction and protocol-aware fuzzing over recovered service surfaces. The evaluation shows that evidence- and feedback-grounded agent coordination improves rehosting success, service recovery, and downstream utility in automated firmware rehosting.

Figures

Figures reproduced from arXiv: 2607.14903 by Fan Zhang, Haitao Xu, Yanbing Shen.

Figure 1
Figure 1. Figure 1: System Architecture of FIRMPILOT • C3–Constrained and Convergent Environment Actuation. Actions can mutate boot scripts, configuration files, and emulator arguments, so deltas must be typed, and the loop must converge under strict budgets without drifting toward unrealistic state injection. These challenges motivate a separation between semantic inference and environment transition. In firmware rehosting, … view at source ↗
Figure 2
Figure 2. Figure 2: Design of the SEARCH Agent Role and Evidence Flow. The SEARCH agent gives the loop a controlled path to outside knowledge. Its query packet is built from firmware metadata, vendor/model/board identifiers, [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 4
Figure 4. Figure 4: Design of the FILE Agent F. NVRAM Agent for Persistent State Synthesis [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figure 3
Figure 3. Figure 3: Design of the PLAN Agent Responsibilities and Feedback. The PLAN agent is the loop control point. It consumes artifact readiness, init and service candidates, probe outcomes, recent error signatures, retrieved snippets, port signals, and accepted deltas, then emits an ordered schedule over FILE, NVRAM, and NETWORK with a failure class and stop/retry condition. The schedule is explicit because persistent-st… view at source ↗
Figure 6
Figure 6. Figure 6: Design of the NETWORK Agent interface conventions into a structured network representation, then selects the smallest evidence-consistent update that im￾proves reachability while preserving application behavior. Inference, Exposure, and Validation [PITH_FULL_IMAGE:figures/full_fig_p005_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: LFwC Ping/Web Success by Firmware Architecture [PITH_FULL_IMAGE:figures/full_fig_p006_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Final-state distribution for the full FIRMPILOT run on LFwC [PITH_FULL_IMAGE:figures/full_fig_p007_8.png] view at source ↗
Figure 10
Figure 10. Figure 10: RQ4 Workflow Scale on 5,256 Web-Success Images [PITH_FULL_IMAGE:figures/full_fig_p008_10.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

40 extracted references · 4 linked inside Pith

  1. [1]

    Towards automated dynamic analysis for linux-based embedded firmware

    D. D. Chen, M. Woo, D. Brumley, and M. Egele, “Towards automated dynamic analysis for linux-based embedded firmware.” inNDSS, 2016

    2016

  2. [2]

    Firmae: Towards large-scale emulation of iot firmware for dynamic analysis,

    M. Kim, D. Kim, E. Kim, S. Kim, Y . Jang, and Y . Kim, “Firmae: Towards large-scale emulation of iot firmware for dynamic analysis,” inProceedings of the 36th Annual Computer Security Applications Conference, 2020, pp. 733–745

    2020

  3. [3]

    Firmguide: Boosting the capability of rehosting embedded linux kernels through model-guided kernel execution,

    Q. Liu, C. Zhang, L. Ma, M. Jiang, Y . Zhou, L. Wu, W. Shen, X. Luo, Y . Liu, and K. Ren, “Firmguide: Boosting the capability of rehosting embedded linux kernels through model-guided kernel execution,” in 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE), 2021, pp. 792–804

    2021

  4. [4]

    Pandawan: quantifying progress in linux-based firmware rehosting,

    I. Angelakopoulos, G. Stringhini, and M. Egele, “Pandawan: quantifying progress in linux-based firmware rehosting,” in33rd USENIX Security Symposium (USENIX Security 24), 2024, pp. 5859–5876

    2024

  5. [5]

    Jetset: Targeted firmware rehosting for embedded systems,

    E. Johnson, M. Bland, Y . Zhu, J. Mason, S. Checkoway, S. Savage, and K. Levchenko, “Jetset: Targeted firmware rehosting for embedded systems,” in30th USENIX Security Symposium, 2021, pp. 321–338

    2021

  6. [6]

    Greenhouse: {Single- Service} rehosting of {Linux-Based} firmware binaries in {User-Space} emulation,

    H. J. Tay, K. Zeng, J. M. Vadayath, A. S. Raj, A. Dutcher, T. Reddy, W. Gibbs, Z. L. Basque, F. Dong, Z. Smithet al., “Greenhouse: {Single- Service} rehosting of {Linux-Based} firmware binaries in {User-Space} emulation,” in32nd USENIX Security Symposium, 2023, pp. 5791–5808

    2023

  7. [7]

    User-space dependency-aware rehosting for linux-based firmware binaries,

    C. Qin, C. Zhang, Y . Zheng, P. Liu, J. Zhang, Y . Li, W. Zhang, Y . Liu, and L. Sun, “User-space dependency-aware rehosting for linux-based firmware binaries,” inProceedings of the Network and Distributed System Security Symposium, 2026

    2026

  8. [8]

    Avatar 2: A multi- target orchestration platform,

    M. Muench, D. Nisi, A. Francillon, and D. Balzarotti, “Avatar 2: A multi- target orchestration platform,” inProceedings 2018 Workshop on Binary Analysis Research, 2018

    2018

  9. [9]

    {HALucinator}: Firmware re-hosting through abstraction layer emulation,

    A. A. Clements, E. Gustafson, T. Scharnowski, P. Grosen, D. Fritz, C. Kruegel, G. Vigna, S. Bagchi, and M. Payer, “ {HALucinator}: Firmware re-hosting through abstraction layer emulation,” in29th USENIX Security Symposium (USENIX Security 20), 2020, pp. 1201– 1218

    2020

  10. [10]

    {P2IM}: Scalable and hardware- independent firmware testing via automatic peripheral interface modeling,

    B. Feng, A. Mera, and L. Lu, “ {P2IM}: Scalable and hardware- independent firmware testing via automatic peripheral interface modeling,” in29th USENIX Security Symposium (USENIX Security 20), 2020, pp. 1237–1254

    2020

  11. [11]

    Fuzzware: Using precise {MMIO} modeling for effective firmware fuzzing,

    T. Scharnowski, N. Bars, M. Schloegel, E. Gustafson, M. Muench, G. Vigna, C. Kruegel, T. Holz, and A. Abbasi, “Fuzzware: Using precise {MMIO} modeling for effective firmware fuzzing,” in31st USENIX Security Symposium, 2022, pp. 1239–1256

    2022

  12. [12]

    QEMU, a fast and portable dynamic translator,

    F. Bellard, “QEMU, a fast and portable dynamic translator,” inUSENIX Annual Technical Conference, FREENIX Track, 2005, pp. 41–46

    2005

  13. [13]

    Sok: Enabling security analyses of embedded systems via rehosting,

    A. Fasano, T. Ballo, M. Muench, T. Leek, A. Bulekov, B. Dolan-Gavitt, M. Egele, A. Francillon, L. Lu, N. Gregoryet al., “Sok: Enabling security analyses of embedded systems via rehosting,” inProceedings of the 2021 ACM Asia Conference on Computer and Communications Security, 2021, pp. 687–701

    2021

  14. [14]

    Toward the analysis of embedded firmware through automated re-hosting,

    E. Gustafson, M. Muench, C. Spensky, N. Redini, A. Machiry, Y . Fratan- tonio, D. Balzarotti, A. Francillon, Y . R. Choe, C. Kruegelet al., “Toward the analysis of embedded firmware through automated re-hosting,” in 22nd International Symposium on Research in Attacks, Intrusions and Defenses (RAID 2019), 2019, pp. 135–150

    2019

  15. [15]

    Karonte: Detecting insecure multi- binary interactions in embedded firmware,

    N. Redini, A. Machiry, R. Wang, C. Spensky, A. Continella, Y . Shoshi- taishvili, C. Kruegel, and G. Vigna, “Karonte: Detecting insecure multi- binary interactions in embedded firmware,” in2020 IEEE Symposium on Security and Privacy (SP), 2020, pp. 1544–1561

    2020

  16. [16]

    Avatar: A framework to support dynamic security analysis of embedded systems’ firmwares

    J. Zaddach, L. Bruno, A. Francillon, D. Balzarottiet al., “Avatar: A framework to support dynamic security analysis of embedded systems’ firmwares.” inNDSS, 2014, pp. 1–16

    2014

  17. [17]

    Retrieval- augmented generation for knowledge-intensive nlp tasks,

    P. Lewis, E. Perez, A. Piktus, F. Petroni, V . Karpukhin, N. Goyal, H. Küttler, M. Lewis, W.-t. Yih, T. Rocktäschelet al., “Retrieval- augmented generation for knowledge-intensive nlp tasks,”Advances in neural information processing systems, vol. 33, pp. 9459–9474, 2020

    2020

  18. [18]

    Chain-of-thought prompting elicits reasoning in large language models,

    J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V . Le, D. Zhouet al., “Chain-of-thought prompting elicits reasoning in large language models,”Advances in neural information processing systems, vol. 35, pp. 24 824–24 837, 2022

    2022

  19. [19]

    Reflex- ion: Language agents with verbal reinforcement learning,

    N. Shinn, F. Cassano, A. Gopinath, K. Narasimhan, and S. Yao, “Reflex- ion: Language agents with verbal reinforcement learning,”Advances in Neural Information Processing Systems, vol. 36, pp. 8634–8652, 2023

    2023

  20. [20]

    CAMEL: Communicative agents for “mind

    G. Li, H. Hammoud, H. Itani, D. Khizbullin, and B. Ghanem, “CAMEL: Communicative agents for “mind” exploration of large language model society,”Advances in Neural Information Processing Systems, vol. 36, pp. 51 991–52 008, 2023

    2023

  21. [21]

    {PentestGPT}: Evaluating and harnessing large language models for automated penetration testing,

    G. Deng, Y . Liu, V . Mayoral-Vilches, P. Liu, Y . Li, Y . Xu, T. Zhang, Y . Liu, M. Pinzger, and S. Rass, “{PentestGPT}: Evaluating and harnessing large language models for automated penetration testing,” in33rd USENIX Security Symposium, 2024, pp. 847–864

    2024

  22. [22]

    Large language model guided protocol fuzzing,

    R. Meng, M. Mirchev, M. Böhme, and A. Roychoudhury, “Large language model guided protocol fuzzing,” inProceedings 2024 Network and Distributed System Security Symposium, 2024

    2024

  23. [23]

    The hitchhiker’s guide to program analysis: A journey with large language models,

    H. Li, Y . Hao, Y . Zhai, and Z. Qian, “The hitchhiker’s guide to program analysis: A journey with large language models,”arXiv preprint arXiv:2308.00245, 2023

    Pith/arXiv arXiv 2023

  24. [24]

    Harnessing the power of llm to support binary taint analysis,

    P. Liu, C. Sun, Y . Zheng, X. Feng, C. Qin, Y . Wang, Z. Li, and L. Sun, “Harnessing the power of llm to support binary taint analysis,”arXiv preprint arXiv:2310.08275, 2023

    Pith/arXiv arXiv 2023

  25. [25]

    Mens sana in corpore sano: Sound firmware corpora for vulnerability research,

    R. Helmke, E. Padilla, and N. Aschenbruck, “Mens sana in corpore sano: Sound firmware corpora for vulnerability research,” inProceedings of the 2025 Network and Distributed System Security Symposium (NDSS), 2025

    2025

  26. [26]

    Device-agnostic firmware execution is possible: A concolic execution approach for peripheral emulation,

    C. Cao, L. Guan, J. Ming, and P. Liu, “Device-agnostic firmware execution is possible: A concolic execution approach for peripheral emulation,” inProceedings of the 36th Annual Computer Security Applications Conference, 2020, pp. 746–759

    2020

  27. [27]

    Automatic firmware emulation through invalidity-guided knowledge inference,

    W. Zhou, L. Guan, P. Liu, and Y . Zhang, “Automatic firmware emulation through invalidity-guided knowledge inference,” in30th USENIX Security Symposium (USENIX Security 21), 2021, pp. 2007–2024

    2021

  28. [28]

    Firmalice-automatic detection of authentication bypass vulnerabilities in binary firmware

    Y . Shoshitaishvili, R. Wang, C. Hauser, C. Kruegel, and G. Vigna, “Firmalice-automatic detection of authentication bypass vulnerabilities in binary firmware.” inNDSS, 2015

    2015

  29. [29]

    Iotfuzzer: Discovering memory corruptions in iot through app-based fuzzing

    J. Chen, W. Diao, Q. Zhao, C. Zuo, Z. Lin, X. Wang, W. C. Lau, M. Sun, R. Yang, and K. Zhang, “Iotfuzzer: Discovering memory corruptions in iot through app-based fuzzing.” inNDSS, 2018, pp. 1–15

    2018

  30. [30]

    Sharing more and checking less: Leveraging common input keywords to detect bugs in embedded systems,

    L. Chen, Y . Wang, Q. Cai, Y . Zhan, H. Hu, J. Linghu, Q. Hou, C. Zhang, H. Duan, and Z. Xue, “Sharing more and checking less: Leveraging common input keywords to detect bugs in embedded systems,” in30th USENIX Security Symposium (USENIX Security 21), 2021, pp. 303–319

    2021

  31. [31]

    AFL++: Combining incremental steps of fuzzing research,

    A. Fioraldi, D. Maier, H. Eißfeldt, and M. Heuse, “AFL++: Combining incremental steps of fuzzing research,” in14th USENIX Workshop on Offensive Technologies (WOOT 20), 2020

    2020

  32. [32]

    Fuzztruction: Using fault injection-based fuzzing to leverage implicit domain knowledge,

    N. Bars, M. Schloegel, T. Scharnowski, N. Schiller, and T. Holz, “Fuzztruction: Using fault injection-based fuzzing to leverage implicit domain knowledge,” in32nd USENIX Security Symposium (USENIX Security 23), 2023, pp. 1847–1864

    2023

  33. [33]

    Drifuzz: Harvesting bugs in device drivers from golden seeds,

    Z. Shen, R. Roongta, and B. Dolan-Gavitt, “Drifuzz: Harvesting bugs in device drivers from golden seeds,” in31st USENIX Security Symposium (USENIX Security 22), 2022, pp. 1275–1290

    2022

  34. [34]

    Charm: Facilitating dynamic analysis of device drivers of mobile systems,

    S. M. S. Talebi, H. Tavakoli, H. Zhang, Z. Zhang, A. A. Sani, and Z. Qian, “Charm: Facilitating dynamic analysis of device drivers of mobile systems,” in27th USENIX Security Symposium (USENIX Security 18), 2018, pp. 291–307

    2018

  35. [35]

    PyPANDA: Taming the PANDAmonium of whole system dynamic analysis,

    L. Craig, A. Fasano, T. Ballo, T. Leek, B. Dolan-Gavitt, and W. Robertson, “PyPANDA: Taming the PANDAmonium of whole system dynamic analysis,” inProceedings 2021 Workshop on Binary Analysis Research, 2021

    2021

  36. [36]

    Sok:(state of) the art of war: Offensive techniques in binary analysis,

    Y . Shoshitaishvili, R. Wang, C. Salls, N. Stephens, M. Polino, A. Dutcher, J. Grosen, S. Feng, C. Hauser, C. Kruegelet al., “Sok:(state of) the art of war: Offensive techniques in binary analysis,” in2016 IEEE Symposium on Security and Privacy (SP), 2016, pp. 138–157

    2016

  37. [37]

    EnIGMA: Enhanced interactive generative model agent for ctf challenges,

    T. Abramovich, M. Udeshi, M. Shao, K. Lieret, H. Xi, K. Milner, S. Jancheska, J. Yang, C. E. Jimenez, F. Khorramiet al., “EnIGMA: Enhanced interactive generative model agent for ctf challenges,”arXiv preprint arXiv:2409.16165, 2024

    Pith/arXiv arXiv 2024

  38. [38]

    Autopatch: Multi-agent framework for patching real-world cve vulnerabilities,

    M. Seo, W. Choi, M. You, and S. Shin, “Autopatch: Multi-agent framework for patching real-world cve vulnerabilities,”arXiv preprint arXiv:2505.04195, 2025

    arXiv 2025

  39. [39]

    Patchpilot: A cost-efficient software engineering agent with early attempts on formal verification,

    H. Li, Y . Tang, S. Wang, and W. Guo, “Patchpilot: A cost-efficient software engineering agent with early attempts on formal verification,” arXiv preprint arXiv:2502.02747, 2025

    Pith/arXiv arXiv 2025

  40. [40]

    PATCHAGENT: A practical program repair agent mimicking human expertise,

    Z. Yu, Z. Guo, Y . Wu, J. Yu, M. Xu, D. Mu, Y . Chen, and X. Xing, “PATCHAGENT: A practical program repair agent mimicking human expertise,” in34th USENIX Security Symposium (USENIX Security 25), 2025, pp. 4381–4400

    2025