{"id":"a9a18184-b870-4f84-b070-d17d679fc9ba","arxiv_id":"2607.16487","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"EM side-channel traces, aligned with dynamic time warping, provide a coverage-like signal that guides black-box firmware fuzzing to higher code coverage than random fuzzing.","lead":"This paper shows that electromagnetic emissions from a running microcontroller can serve as a feedback signal to guide black-box firmware fuzzing. If valid, it would let testers find bugs in devices whose firmware cannot be extracted, instrumented, or accurately emulated.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The evaluation confounds EM-derived feedback with the seed-selection scheduler; an ablation with a control tree is needed to attribute coverage gains to EM side-channel information.","rationale":"I read the paper in good faith and acknowledge its strengths: a clear motivation, a plausible signal-processing pipeline, a tree-based indexing scheme that addresses scalability, and an evaluation across multiple real firmware targets. The reader's weakest_assumption focused on the DTW divergence detector parameters and the fidelity of the custom QEMU model. Those are legitimate concerns, but I do not see them as the single most load-bearing issue. The DTW parameters, though unspecified, could be tuned empirically; the QEMU model, though custom, provides a relative comparison where any model inaccuracy might affect both modes similarly. The more fundamental problem is that the comparison between 'EM-guided, tree-aware selection' and 'random unguided selection' changes two things at once. The paper's own framing in §5.2.1 says 'EM-guided selection exceeds the final instruction and edge coverage of random selection' and attributes this to 'EM-derived feedback.' However, the EM-guided arm's advantage includes a sophisticated scheduler that the random arm entirely lacks. The selection strategies in §4.4 (most descendants, most immediate children, frontier leaves, random) plus UCB are likely to improve exploration even if the tree nodes were assigned arbitrarily, because they avoid repeatedly mutating the same seed and systematically probe under-explored regions. A proper control would keep the scheduler fixed and vary only the source of the tree—EM-derived divergence versus a random/trivial grouping. Without that ablation, the experiment cannot distinguish 'EM signals help' from 'a structured scheduler helps.' This is a concrete, falsifiable concern, not a matter of taste, and it directly affects the central claim. I would still not reject the paper outright, because the method may well work; rather, the verdict should remain conditional, with the added condition that the authors run the control-tree ablation and report repeated trials with variance. Secondary issues—missing the EM-guided row for Soldering Station in Table 3, unspecified τ and L, and apparent single-run evaluations—reinforce the need for revision but do not change my verdict class.","tokens_in":17226,"tokens_out":4480,"duration_ms":55678,"concrete_test":"Run an ablation on at least two targets (e.g., GPS and CNC) where the EM-derived divergence tree is replaced by a control tree with the same node count and depth but with nodes assigned by a random hash of the input (or by input length), keeping all four node-selection strategies and the UCB scheduler exactly as in §4.4. Also run a second control using the same mutations with a simple round-robin seed selector (no tree). Compare final instruction/edge coverage (§5.2.1, Table 1, Fig. 7) and novelty hit-rate (§5.2.2, Table 3). If the control tree or round-robin achieves coverage gains comparable to the EM-guided mode, the reported improvements are not attributable to EM side-channel information; if EM-guided substantially outperforms both controls, the central claim is supported.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's central claim is that EM-derived feedback, specifically the divergence tree built from DTW-based trace comparison, causes the observed coverage improvements. But the experiment in §5.2 does not isolate this causal factor. The EM-guided mode uses not only the EM-based divergence tree but also four complementary node-selection strategies (most descendants, most immediate children, frontier leaves, random) combined with an ε-greedy + UCB scheduler (§4.4). The baseline 'random unguided selection' uses only uniform random seed selection. Thus the treatment differs in two variables simultaneously: (1) the availability of EM-derived divergence information and (2) the entire node-selection/scheduling policy. The observed gains could plausibly be due to the scheduler alone—e.g., prioritizing frontier leaves or high-out-degree nodes naturally diversifies exploration, even if the tree topology carries no real information about control flow. If a 'fake' tree with arbitrary node assignments, or even a round-robin selector over seeds, yields similar coverage growth and novelty hit-rates, the EM signal is not the load-bearing component. This is not merely a matter of missing error bars; it is an internal-validity confound that directly undermines the claimed contribution. Secondary issues (unspecified DTW parameters τ and L in §4.2, missing EM-guided row for Soldering Station in Table 3, no repeated runs) further weaken the evaluation, but the confound is the more serious threat to the central claim.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Fuzz'EMup, a black-box embedded-firmware fuzzer that uses EM side-channel emanations as a coverage proxy. The pipeline selects an informative carrier band via an idle/active Welch t-statistic, converts each trace to a band-limited envelope, aligns traces with banded DTW, and detects sustained divergence from the DTW offset profile. Executions are organized in a divergence tree to keep comparison costs sublinear. A scheduler combining four node-selection strategies with an ε-greedy + UCB bandit chooses which seeds to mutate. The evaluation runs four real firmware targets on an STM32L476RG board, compares EM-guided selection against uniform random selection, and measures coverage by replaying hardware inputs in a custom QEMU model. The paper claims that EM-derived feedback yields higher instruction and edge coverage, reaches baseline coverage with fewer inputs, and produces a higher novelty hit-rate than unguided fuzzing.","tokens_in":17639,"tokens_out":4445,"duration_ms":51174,"significance":"If the causal claim is confirmed, this would be a meaningful step for black-box embedded fuzzing: guidance from a physical side channel without binary extraction, instrumentation, or accurate emulation, with a scalable trace-indexing scheme. The paper has real strengths: it uses physical hardware and EM capture, it attempts an independent QEMU-based coverage ground truth, and it keeps mutation operators and initial seeds matched between modes. The divergence-tree idea is a sensible answer to the scalability challenge. However, the current evidence does not yet isolate the EM feedback from the scheduler, and the load-bearing DTW parameters are undisclosed. These issues are fixable with additional experiments and disclosures, but they are central to the paper's claim.","major_comments":[{"comment":"The treatment differs from the control in two variables simultaneously. EM-guided mode consists of the DTW divergence tree plus the ε-greedy/UCB scheduler and four node-selection strategies; the 'random unguided' baseline is uniform random seed selection without any tree. The observed coverage gains could be due entirely to the scheduler, e.g., the frontier-leaf and most-descendants heuristics diversify exploration even if the tree topology carried no EM information. The statement in §5.2.2 that mutation operators, initial seeds, and budget are identical does not isolate the EM signal. Please add a control condition in which the same scheduler runs on a tree whose node assignments are random or on a round-robin selector over the same nodes, and show that EM-derived topology is necessary for the gains.","section":"§5.2 and §4.4"},{"comment":"The offset-drift detector is load-bearing, but τ, L, the Sakoe–Chiba half-width W, the divergence-time proximity window, and the scheduler hyperparameters ε, c, α, λ, k are never specified or justified. The text says L is 'small' and W is 'large enough to cover typical jitter with a safety margin,' which is not a calibration procedure. If τ and L were chosen using the same targets' QEMU coverage, the evaluation is circular; if chosen a priori, report values and sensitivity. Without this, the claimed reliability of the 'sustained divergence' signal cannot be assessed or reproduced.","section":"§4.2–§4.4"},{"comment":"The empirical basis is a single run per mode. No repeated runs, variance, or confidence intervals are reported, and Table 3 omits the EM-guided row for Soldering Station — the only target for which the 'consistent' claim is therefore not directly shown. The CNC unique-instruction gap (989 vs. 8) is large but, without variance or a complete table, could be a stochastic artifact or a reporting omission. Please report all rows and provide repeated runs or otherwise justify that the observed gap cannot be attributed to randomness.","section":"§5.2, Table 3"},{"comment":"Coverage ground truth comes from replaying hardware inputs in a custom QEMU STM32L476RG model. No validation of the model's fidelity is given (e.g., matching UART/DMA/timer behavior or instruction traces on a known input). If the model diverges from hardware, all coverage numbers and therefore the central comparison are unreliable. Please validate the model, or at minimum document the modeled peripherals and show that representative inputs produce the same control flow as on hardware.","section":"§5.1"}],"minor_comments":[{"comment":"In the Challenge 2 heading, 'divregence' should be 'divergence'.","section":"§4"},{"comment":"'Welcht' should be 'Welch'. Equation (1) is also typeset incorrectly: 'µact(f)−µ idle(f)s' appears garbled and the denominator needs parentheses and a radical.","section":"§4.1"},{"comment":"The 'microsecond-scale window' for divergence proximity is described qualitatively. A concrete value or derivation would improve reproducibility.","section":"§4.3"},{"comment":"The legend uses 'tree' where 'EM-guided' is clearer, and the y-axis scales differ across subplots. Please state this explicitly or use normalized axes so cross-target comparisons are not misleading.","section":"Figure 7"},{"comment":"Reference [4] contains the placeholder 'accessed: YYYY-MM-DD' and should be completed.","section":"References"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth a look, but the evaluation has a load-bearing confound. The idea is real: use EM traces, DTW alignment, and offset-drift to detect program divergence, then organize executions in a divergence tree to fuzz black-box MCU firmware. That specific mechanism—DTW offset drift as a divergence signal plus a divergence tree—is new relative to the power-trace/ML work they cite. The pipeline is coherent: band selection via Welch t-statistic, demodulation, execution-window detection, then banded DTW. The divergence-tree indexing argument is plausible and a real engineering contribution.\n\nWhat the paper does well: evaluation on four real firmware targets on real hardware, with QEMU replay for ground-truth coverage, not just crash detection. The reported gains are consistent and sometimes large (CNC instruction coverage 4682 vs 3701). Working without binary extraction, instrumentation, or emulation is a meaningful step for black-box IoT fuzzing.\n\nNow the soft spots, in order. First, the main confound. The treatment is EM-guided tree-aware selection with four node-selection strategies and an ε-greedy + UCB scheduler; the baseline is uniform random seed selection. Two variables change at once. The scheduler alone—prioritizing frontier leaves, high-out-degree nodes, most descendants—might drive exploration even if the tree topology carried no real control-flow information. Without an ablation using a fake tree or arbitrary node assignments, the gains cannot be attributed to the EM-derived feedback. This is the issue that directly threatens the central claim.\n\nSecond, τ and L in §4.2 are never specified or justified. They are load-bearing for divergence detection. Saying L is 'small' and τ is 'a threshold' is not reproducible. Third, no repeated runs or error bars. The CNC jump from 8 to 989 unique instructions is large enough that single-run noise seems unlikely to explain it, but I can't rule it out. Fourth, Table 3 is missing the EM-guided row for Soldering Station; the 1.6× novelty hit-rate claim is not fully documented without that row. Minor, but sloppy.\n\nThe paper deserves a serious referee. The method is novel, clearly written, and the confound is fixable with the right ablation and parameter disclosure. I would accept only after those are addressed.","headline":"A genuinely new EM-side-channel fuzzing feedback path, but the evaluation doesn't isolate the EM signal from the scheduler, so the headline claim is not yet proven.","tokens_in":18086,"tokens_out":2414,"would_cite":true,"duration_ms":24463,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that electromagnetic side-channel traces can reliably guide black-box firmware fuzzing as a coverage proxy, without needing binary extraction, instrumentation, or accurate emulation.","keywords":["embedded firmware fuzzing","EM side-channel","black-box fuzzing","coverage proxy","dynamic time warping","divergence tree","IoT security","fuzzing guidance"],"falsifier":"Replay a set of inputs where the true execution path is known—some pairs that definitely execute identical code and some that definitely diverge—and test whether the sustained-drift criterion with the paper's chosen L and τ separates them without false positives or false negatives. The paper does not report such a calibration experiment, nor does it specify concrete values for τ and L; an independent run that fails to separate known same-path and different-path traces would disprove the coverage proxy.","tokens_in":17164,"feed_emoji":"📡","tokens_out":4443,"duration_ms":48534,"temperature":0.7,"pith_summary":"The paper tries to establish that a fuzzer can use electromagnetic (EM) side-channel traces as a coverage proxy when firmware cannot be extracted, instrumented, or accurately emulated. Because raw EM traces are noisy and timing-jittered, the authors filter each trace to a frequency band with strong idle-to-active contrast, align traces with dynamic time warping, and detect where the warping offset begins drifting persistently—the point where control flow diverges. These divergence points are organized in a tree so new inputs are compared only against a few representative traces, keeping the comparison cost scalable. On four real firmware targets, EM-guided selection reaches higher instruction and edge coverage with fewer inputs than unguided random selection, and discovers code regions random fuzzing misses. If true, this makes coverage-like guidance available in settings previously limited to inefficient black-box fuzzing.","feed_headline":"EM traces guide black-box fuzzing to more code paths","feed_subtitle":"No firmware extraction needed: EM-derived divergence finds higher coverage with fewer inputs than unguided fuzzing.","key_machinery":"The central mechanism is the DTW-based divergence detector. After band-pass filtering and demodulating the EM trace into an activity envelope, dynamic time warping—a standard alignment that stretches or compresses time axes to match similar signals—aligns a candidate trace to a node representative. The offset profile offset[k]=i_k−j_k is monitored across the warping path, and a sustained change in its local mean, computed with a pre-window and post-window of length L and threshold τ, marks the divergence time. A divergence tree organizes traces by these divergence times so each insertion aligns only along one root-to-leaf path, and a multi-armed bandit scheduler (ε-greedy with Upper Confiden","core_discovery":"The central claim is that sustained drift in the dynamic time warping offset between an EM trace and a previously observed reference trace marks a genuine control-flow divergence, and that using these divergence times as a coverage proxy can guide a black-box fuzzer. On four bare-metal microcontroller firmware targets spanning parser, control-loop, and DMA-driven code, EM-guided selection outperforms random selection under identical mutation operators: higher final instruction and edge coverage, lower input count to reach baseline coverage, and substantially higher novelty hit-rate, defined as the fraction of inputs reaching previously unseen instructions or edges. The method never observes","pith_inferences":["An obvious extension the authors leave implicit is testing whether EM-guided coverage actually translates into more discovered bugs; injecting known vulnerabilities and measuring detection rates would connect the coverage gains to security outcomes.","The method uses only an external probe and a serial bridge, so it could plausibly be combined with conventional black-box signals such as resets, timeouts, or output codes to build a fully unattended black-box fuzzer; the paper deliberately omits those signals from the guidance.","The divergence tree could serve as a state machine for regression testing or replay: inputs that branch at the same point form equivalence classes, which might help deduplicate failures and triage crashes in black-box settings.","The one-time frequency-band and probe-position calibration could likely be automated, reducing setup effort and making the method easier to deploy across different boards."],"forward_implications":["If EM guidance works as reported, coverage-like fuzzing becomes possible on devices with read-out protection, encrypted firmware, or proprietary cores, where existing coverage methods are unavailable.","The divergence tree keeps per-input comparison cost roughly constant as the corpus grows, making the guidance scalable to long fuzzing campaigns.","Because the same mutation operators and initial seeds are used in both modes, the measured coverage gains are attributable to seed selection driven by EM divergence, not to differences in mutation.","The paper reports that inputs discarded by the divergence tree, when replayed separately, add no additional coverage; this suggests the divergence criterion is not over-eagerly accepting redundant inputs.","The approach is application-agnostic and depends mainly on EM emission sensitivity, so it should extend to other microcontroller firmware beyond the four evaluated targets."],"fun_headline_variants":["EM side-channel feedback boosts black-box fuzzer coverage","Fuzzing without firmware: EM traces lead to deeper coverage","Electromagnetic leaks guide fuzzers to undiscovered code paths","EM emanations replace coverage guidance in black-box fuzzing"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that the DTW offset-drift detector, with its unspecified window length L and threshold τ, reliably separates genuine control-flow divergence from noise and jitter, and that the rehosted emulator used to measure ground-truth coverage faithfully reproduces hardware execution; if either gives way the central claim collapses.","fun_headline_variants_meta":{"raw":{"variants":["EM side-channel feedback boosts black-box fuzzer coverage","Fuzzing without firmware: EM traces lead to deeper coverage","Electromagnetic leaks guide fuzzers to undiscovered code paths","EM emanations replace coverage guidance in black-box fuzzing"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00092,"raw_usage":{"total_tokens":3765,"prompt_tokens":711,"completion_tokens":3054,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":455,"completion_tokens_details":{"reasoning_tokens":2993}},"tokens_in":455,"tokens_out":3054,"duration_ms":24971,"temperature":1.0,"reasoning_tokens":2993,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T20:47:47.564087+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Replay a set of inputs where the true execution path is known—some pairs that definitely execute identical code and some that definitely diverge—and test whether the sustained-drift criterion with the paper's chosen L and τ separates them without false positives or false negatives. The paper does not report such a calibration experiment, nor does it specify concrete values for τ and L; an independent run that fails to separate known same-path and different-path traces would disprove the coverage proxy.","supporting_citations":[],"review_version":1}