{"id":"d9b3067b-ccba-4386-85f5-cf79260a7eb9","arxiv_id":"2606.24549","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":7.0,"correctness_risk":"high","formal_verification":"none","parameter_count":0,"one_line_summary":"FirmCure uses LLM-based adaptive perception, reflective synthesis, and autonomous runtime intervention to rehost 21 firmware images at 100% port opening and 90.5% interactivity rates across 5 architectures.","lead":"FirmCure is an LLM-driven framework with three modules for automatically rehosting Linux firmware on custom hardware. It could reduce expert effort needed to analyze security of IoT devices.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"LLM reliability for static dependency extraction and runtime remediation is the load-bearing assumption","rationale":"Reader's weakest_assumption directly identifies the LLM-centric modules as the unverified core; full-text evaluation details would be needed to lower the risk, but the abstract-level claim leaves this exact gap exposed.","tokens_in":1716,"tokens_out":295,"duration_ms":11700,"concrete_test":"Re-execute the framework on the 21 images using an independent static-analysis baseline (e.g., angr or Ghidra scripts) for dependency extraction and a non-LLM rule-based remediation script; compare success rates and count cases where the LLM version diverges or requires manual correction.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"FirmCure's headline results (100% port opening, 90.5% interactivity) rest on three LLM modules: Adaptive Perception Inference (static analysis for structural dependencies), Reflective Synthesis (iterative config optimization), and Autonomous Runtime Intervention (real-time fault diagnosis). These must succeed without missing hardware-specific behaviors across 5 architectures or introducing false positives. Firmware static analysis routinely fails on peripheral initialization, memory-mapped I/O, and vendor-specific boot sequences; LLMs add hallucination risk on binary or decompiled code. The paper reports generalization but supplies no ablation on LLM prompt sensitivity, error rates per module, or cases where hardware emulation was still required despite the autonomy claim.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper introduces FirmCure, the first LLM-driven full-system rehosting framework for Linux-based IoT firmware. It comprises three modules: Adaptive Perception Inference to extract structural dependencies via static analysis, Reflective Synthesis for iterative configuration optimization, and Autonomous Runtime Intervention for real-time fault diagnosis and error remediation. On 21 firmware images from 10 vendors across 5 architectures, it reports 100% network port opening and 90.5% service interactivity, outperforming baselines, with intervention strategies that generalize and enable reproduction of known vulnerabilities plus discovery of new flaws.","tokens_in":1842,"tokens_out":536,"duration_ms":22349,"significance":"If the empirical results and LLM-module reliability hold under scrutiny, FirmCure could meaningfully advance scalable firmware security analysis by reducing expert intervention in rehosting heterogeneous devices, enabling broader vulnerability research on custom IoT hardware.","major_comments":[{"comment":"Evaluation section: aggregate success rates (100% port opening, 90.5% interactivity) are reported without firmware selection criteria, baseline implementation details, error bars, or confirmation that post-hoc adjustments were avoided, rendering the central performance claims unverifiable from the provided text.","section":"Evaluation"},{"comment":"Adaptive Perception Inference and Autonomous Runtime Intervention modules: the load-bearing assumption that the LLM components reliably extract dependencies and perform remediation without missing hardware-specific behaviors (e.g., peripheral initialization, memory-mapped I/O, vendor boot sequences) across 5 architectures lacks supporting ablations on prompt sensitivity, per-module error rates, or false-positive rates in vulnerability reproduction.","section":"§3"},{"comment":"Generalization claim: the assertion that intervention strategies generalize across heterogeneous firmware is not accompanied by case studies or metrics showing instances where static analysis failed or hardware emulation remained necessary despite the autonomy claim.","section":"Experiments"}],"minor_comments":[{"comment":"Clarify the specific LLMs, versions, and prompt templates used in each module to aid reproducibility.","section":"§3"},{"comment":"Add a table comparing per-firmware results against each baseline rather than aggregate figures only.","section":"Evaluation"},{"comment":"Ensure all architecture-specific handling details are explicitly described rather than summarized.","section":null}],"recommendation":"major_revision","confidential_remarks":"The evaluation appears preliminary with limited transparency on experimental controls; addressing reproducibility (e.g., via code release) would strengthen the submission for this venue."},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive and detailed feedback. We address each major comment point by point below, indicating where revisions will be made to strengthen the manuscript.","responses":[{"response":"We acknowledge that the current manuscript reports aggregate success rates without providing firmware selection criteria, detailed baseline implementation steps, error bars, or explicit confirmation that post-hoc adjustments were avoided. This limits verifiability of the central claims. In the revised manuscript, we will expand the Evaluation section to include these elements: explicit selection criteria for the 21 firmware images, implementation details for all baselines, statistical error bars on the reported rates, and a statement confirming no post-hoc adjustments were performed.","revision_made":"yes","referee_comment":"[Evaluation] Evaluation section: aggregate success rates (100% port opening, 90.5% interactivity) are reported without firmware selection criteria, baseline implementation details, error bars, or confirmation that post-hoc adjustments were avoided, rendering the central performance claims unverifiable from the provided text."},{"response":"The referee correctly notes the absence of ablations on prompt sensitivity, per-module error rates, and false-positive rates for vulnerability reproduction. The manuscript relies on overall success metrics to support LLM reliability but does not provide these supporting analyses. We will add a dedicated subsection (or expand §3 and the evaluation) with prompt sensitivity experiments, per-module breakdown of success/error rates, and analysis of false positives in vulnerability reproduction to address this gap.","revision_made":"yes","referee_comment":"[§3] Adaptive Perception Inference and Autonomous Runtime Intervention modules: the load-bearing assumption that the LLM components reliably extract dependencies and perform remediation without missing hardware-specific behaviors (e.g., peripheral initialization, memory-mapped I/O, vendor boot sequences) across 5 architectures lacks supporting ablations on prompt sensitivity, per-module error rates, or false-positive rates in vulnerability reproduction."},{"response":"We agree that the generalization claim requires more concrete supporting evidence. The manuscript asserts that intervention strategies generalize but does not include case studies or quantitative metrics on cases where static analysis failed or where hardware emulation steps remained necessary. In the revision, we will add case studies and associated metrics illustrating such instances to substantiate the autonomy and generalization claims.","revision_made":"yes","referee_comment":"[Experiments] Generalization claim: the assertion that intervention strategies generalize across heterogeneous firmware is not accompanied by case studies or metrics showing instances where static analysis failed or hardware emulation remained necessary despite the autonomy claim."}],"tokens_in":1383,"tokens_out":542,"duration_ms":27044,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main point is that FirmCure introduces LLM modules for fully autonomous Linux firmware rehosting and claims 100% network port opening plus 90.5% service interactivity across 21 images from 10 vendors and 5 architectures. That is the headline result.\n\nWhat is new is the specific three-part design: Adaptive Perception Inference to pull structural dependencies from static analysis, Reflective Synthesis for iterative config tuning, and Autonomous Runtime Intervention for runtime fault diagnosis and fixes. Earlier rehosting work needed heavy expert input on custom hardware and boot sequences, so framing the problem this way is a clear shift.\n\nThe paper does a reasonable job showing coverage across heterogeneous firmware and stating that the intervention strategies generalize. It also reports that the system reproduces known vulnerabilities and finds new ones, which is the end goal for security analysis.\n\nThe soft spots are in the evaluation. The abstract gives only aggregate rates with no firmware selection criteria, no baseline code or implementation details, no error bars, and no per-module breakdown or ablation on prompt sensitivity. The load-bearing assumption that the LLM modules can reliably handle static dependency extraction and real-time remediation without missing hardware-specific behaviors or introducing false positives is not tested in the provided text. Firmware static analysis often fails on peripheral init and memory-mapped I/O, and LLMs add hallucination risk on decompiled code, so the stress-test concern holds up from what is shown.\n\nThis is for people working on automated IoT firmware analysis. A reader focused on LLM applications to systems problems would find the module descriptions useful, but anyone planning to rely on the performance numbers would need the full experimental section first.\n\nIt deserves peer review because the problem is real and the direction is fresh, even though the current evidence is not yet strong enough to stand on its own.","headline":"FirmCure is the first LLM-driven rehosting system with those three named modules and reports high success on 21 samples, but the evaluation details are too thin to verify the central claims.","tokens_in":2321,"tokens_out":446,"would_cite":false,"duration_ms":27709,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"FirmCure uses LLMs to autonomously rehost Linux firmware by extracting dependencies, optimizing configs, and fixing runtime errors.","keywords":["firmware rehosting","IoT security","LLM automation","full-system emulation","vulnerability analysis","embedded device testing","runtime intervention","adaptive configuration"],"falsifier":"A firmware image from a new vendor or architecture where FirmCure produces incorrect dependency graphs or fails to open network ports because of missed hardware behaviors.","tokens_in":2625,"feed_emoji":"🔧","tokens_out":724,"duration_ms":17930,"temperature":0.7,"pith_summary":"The paper introduces FirmCure as a framework that automates the full-system rehosting of Linux-based firmware for custom IoT devices. Existing methods require heavy expert intervention to handle specialized architectures and hardware configurations, creating bottlenecks in security analysis. FirmCure combines static analysis with LLM modules for dependency extraction, iterative configuration, and real-time error fixes. On 21 firmware images from 10 vendors and 5 architectures it reached 100 percent network port opening and 90.5 percent service interactivity while reproducing known vulnerabilities and finding new ones. A sympathetic reader would care because successful rehosting is a prerequisite for practical vulnerability work on embedded systems that are otherwise hard to test.","feed_headline":"LLM framework rehosts Linux firmware at 100% port success","feed_subtitle":"Achieves full network access and 90.5% service interactivity on 21 IoT images from 10 vendors without expert intervention.","key_machinery":"The Adaptive Perception Inference mechanism for dependency extraction combined with Reflective Synthesis for configuration optimization and Autonomous Runtime Intervention for runtime error remediation.","core_discovery":"FirmCure is the first LLM-driven full-system rehosting framework that uses an Adaptive Perception Inference mechanism to extract firmware structural dependencies via static analysis, a Reflective Synthesis module for iterative configuration optimization, and an Autonomous Runtime Intervention module for real-time error remediation through runtime fault diagnosis and monitoring. On 21 IoT firmware images from 10 vendors across 5 architectures it achieved a 100 percent network port opening rate and 90.5 percent service interactivity, substantially outperforming baselines, with intervention strategies that generalize across heterogeneous firmware and that reproduce known vulnerabilities while d","pith_inferences":["The same LLM-driven loop might reduce manual work in rehosting non-Linux embedded systems if the static analysis step is extended.","Combining the runtime intervention module with hardware-in-the-loop testing could catch behaviors the current static extraction misses.","Patterns found across the 21 tested images could seed a shared library of common hardware dependency fixes for future devices.","If the approach scales, routine firmware updates could be checked for security issues in an automated pipeline."],"forward_implications":["Security analysis of custom IoT devices becomes possible without manual expert configuration for each device.","Known vulnerabilities can be reproduced at scale across multiple architectures.","New security flaws can be discovered in firmware that previously could not be rehosted.","Intervention strategies learned on one set of devices transfer to other heterogeneous firmware.","The rehosting process completes with full network access and high service interactivity rates."],"fun_headline_variants":["FirmCure achieves 100% port success on Linux firmware rehosting","LLM rehoster reaches 90.5% interactivity on 21 IoT images","Autonomous rehosting generalizes across 5 firmware architectures","FirmCure succeeds on 21 images from 10 vendors without experts"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The LLM modules can reliably extract accurate firmware structural dependencies via static analysis and perform correct real-time error remediation without missing hardware-specific behaviors or introducing false positives.","fun_headline_variants_meta":{"raw":{"variants":["FirmCure achieves 100% port success on Linux firmware rehosting","LLM rehoster reaches 90.5% interactivity on 21 IoT images","Autonomous rehosting generalizes across 5 firmware architectures","FirmCure succeeds on 21 images from 10 vendors without experts"]},"model":"grok-4.3","cost_usd":0.006785,"raw_usage":{"total_tokens":3171,"prompt_tokens":699,"num_sources_used":0,"completion_tokens":78,"cost_in_usd_ticks":67849500,"prompt_tokens_details":{"text_tokens":699,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":2394,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":699,"tokens_out":78,"duration_ms":17329,"temperature":1.0,"reasoning_tokens":2394,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-25T23:13:57.885373+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"A firmware image from a new vendor or architecture where FirmCure produces incorrect dependency graphs or fails to open network ports because of missed hardware behaviors.","supporting_citations":[],"review_version":1}