{"id":"1c8d2389-43d8-4529-9804-fb7a2f41bec6","arxiv_id":"2508.21606","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"A dual detection scheme combining a timing threshold and a Random Forest classifier is proposed to detect simulated AES anomalies, but the paper gives no quantitative results and its features leak the injected anomalies.","lead":"This paper proposes a monitoring system that uses a timing threshold and a Random Forest classifier to flag encryption blocks that were altered by injected delays or bit flips on a PYNQ FPGA board. The authors claim the machine-learning detector is more accurate than the threshold, but the report does not include any measured accuracy numbers.","discovery_kind":"incremental","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Detection is tautological: features include injected delay and modified plaintext byte, so the ML-vs-threshold comparison does not measure side-channel detection.","rationale":"The reader's verdict is REJECT, with the weakest assumption stated as the unrealism of simulated anomalies. My concern is related but more specific: even within the synthetic protocol, the features include the injected markers, making the detection tautological. The reader's rationale does mention this ('the classifier can trivially detect the injection markers'), so there is overlap, but the reader's formal weakest_assumption focuses on external validity rather than this internal construct-validity flaw. The proposed concrete test would settle whether the ML advantage disappears when the markers are excluded. If it does, the central claim is unsupported regardless of the realism of the anomalies. Since this strengthens rather than contradicts the reader's REJECT, the verdict remains unchanged.","tokens_in":3652,"tokens_out":5904,"duration_ms":67599,"concrete_test":"Re-run the entire evaluation with two changes: (1) measure encryption time excluding the artificial sleep (e.g., timestamp immediately before and after the sleep and subtract the sleep interval), and (2) remove the plaintext bytes (or specifically the first byte) from the feature vector, using only the corrected timing and ciphertext bytes. If the Random Forest and threshold accuracies then fall to near chance, the original results are artifacts of reading the injection markers. Alternatively, validate on a public real side-channel dataset, such as Bernstein's cache-timing traces or DFA fault traces, and report detection accuracy.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that a Random Forest detector outperforms static timing thresholds for side-channel attack detection. The load-bearing weakness is that the evaluation protocol makes detection trivial and not representative of real attacks. In Section 2.1, anomalies are injected as a 5–20 ms sleep before encryption and an XOR of the plaintext's first byte with 0xFF. In Algorithm 2 (Steps 2–3), the recorded encryption time t_i is measured after the sleep, so the artificial delay is directly embedded in the timing feature. Moreover, Step 3 uses 'original block bytes' as ML features, which include the modified plaintext byte for fault-injected blocks. Thus the classifier and the threshold detector are reading the injection markers themselves. Even if quantitative results were reported, the comparison would show only that the systems can detect the manually inserted delay/bit-flip, not genuine side-channel leakage. This is a construct-validity failure: the experiment does not test what the abstract and introduction claim. Without removing these injection markers from the features, or validating against real side-channel traces (e.g., cache-timing or voltage-glitch fault attacks), the reported superiority of ML over thresholds is unsupported.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a dual detection framework for AES-128 encryption anomalies on CPU and PYNQ-Z1 SoC platforms. Anomalies are simulated by injecting a 5–20 ms delay before encryption and by XORing the first plaintext byte with 0xFF. The authors compare a statistical threshold detector based on encryption time with a Random Forest classifier trained on timing and per-block byte features, claiming that the ML approach outperforms static thresholds in accuracy and maintains real-time feasibility. The manuscript describes Algorithms 1 and 2, the experimental setup, and qualitative deployment results, but it reports no quantitative evaluation: no accuracy, false-positive rate, confusion matrix, plot, or dataset statistics are presented.","tokens_in":3938,"tokens_out":2031,"duration_ms":25497,"significance":"If the central claim were substantiated, a lightweight, non-intrusive Random Forest detector that flags anomalous AES blocks on an embedded PYNQ-Z1 with better accuracy than a timing threshold would be a useful contribution for low-power systems lacking hardware performance counters. The paper also names reproducible ingredients: a simple threshold rule, a Random Forest classifier, and a concrete embedded platform. However, the evaluation protocol is construct-invalid for side-channel attack detection: the features directly encode the injected delay and bit flip, making the detection task trivial. In addition, no quantitative results are given. The potential significance is therefore not realized in the present manuscript.","major_comments":[{"comment":"The central claim that \"the ML approach outperforms static thresholds in accuracy\" is asserted without any quantitative evidence. The section reports no accuracy numbers, false-positive rates, confusion matrices, ROC curves, or timing distributions. It also gives no resource-utilization figures despite claiming \"under 30% of FPGA resources.\" Without numeric results and a description of the evaluation protocol, the main comparative conclusion is unsupported.","section":"Section 3 (Deployment and Evaluation)"},{"comment":"The evaluation is tautological. In Step 2, the encryption time t_i is recorded after an injected 5–20 ms sleep for delay anomalies, so the injected delay is directly embedded in the timing feature. In Step 3, the feature set includes \"original block bytes,\" which for fault-injected blocks contain the first byte XORed with 0xFF. Thus both the threshold and the Random Forest are reading the injection markers themselves. The experiment does not test detection of a side-channel attack; it tests detection of manually inserted delays and plaintext modifications. This construct-validity failure makes the claimed ML-vs-threshold comparison uninformative for the stated purpose.","section":"Algorithm 2, Steps 2–3 and Section 2.1"},{"comment":"The threshold T = μ + 3×((max−min)/N) is computed on the same dataset that is then evaluated, and no separate calibration set is described. This is an in-sample evaluation and will inflate the apparent performance of the threshold method. The metric comparison between ML and threshold is therefore not meaningful unless a proper train/validation/test split is applied to the threshold as well as the classifier.","section":"Algorithm 2, Step 5; Section 2.2"},{"comment":"The simulated anomalies are not validated as proxies for real timing and fault-injection attacks. The injected 5–20 ms delay is many orders of magnitude larger than realistic timing side-channels (e.g., cache-timing differences on AES), and the fault model corrupts the plaintext byte rather than intermediate cipher state as in differential fault analysis. The paper does not test against real side-channel traces or show that detection transfers. The claimed generalizability to \"timing and fault injection attacks\" is therefore unsupported.","section":"Section 2.1 and Abstract"}],"minor_comments":[{"comment":"The feature description is ambiguous: the abstract and introduction mention \"ciphertext features,\" while Step 3 says \"original block bytes.\" Since the original block bytes include the injected bit flip, this ambiguity is not merely cosmetic; the authors should clarify exactly which bytes are used as ML inputs.","section":"Algorithm 2, Step 3"},{"comment":"The threshold rule is described as \"inspired by the empirical 3-sigma rule,\" but the formula T = μ + 3×(max−min)/n is not the 3-sigma rule and no statistical justification is given. If this rule is retained, its distributional properties and calibration should be discussed.","section":"Section 2.2"},{"comment":"The claim that \"the Random Forest classifier... improved accuracy with inference latency under 5 ms\" is not supported by any measured latency plot or table. Similarly, the FPGA resource claim needs a utilization report or a synthesis summary.","section":"Section 3"},{"comment":"The manuscript contains no tables or figures with experimental data. Figure 1 is referenced but not described in the text. The evaluation section is a high-level summary rather than a report of results; concrete numbers are essential for a comparative study.","section":"General"},{"comment":"Some references are incomplete or lack venue/page information (e.g., [1], [9], [10]). Please verify and complete the bibliography.","section":"References"}],"recommendation":"reject","confidential_remarks":"The paper is a workshop-style extended abstract rather than a complete evaluation study. The main result is not merely under-reported; the experimental design makes the detection task trivial by putting the injected anomaly directly into the feature set. Rebuttal would require a fundamentally different evaluation using real side-channel traces or at least features that do not contain the injection markers, plus a full quantitative comparison. This is beyond the scope of a minor revision and, in the current form, the central claim cannot be accepted."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a framework description with no quantitative results, and the one comparative claim in the abstract is undermined by a construct-validity problem. The good parts: the setup is transparent and potentially portable. The authors describe a concrete AES-128-ECB target on a PYNQ-Z1, a statistical threshold rule, and a Random Forest, and they are upfront about simulating anomalies with delays and plaintext bit flips. If they had shipped code, data, and standard metrics, this could serve as a useful starting point for lightweight anomaly monitoring on that board.\n\nThe soft spots are large, though. Most importantly, there are no numbers anywhere in the manuscript. No accuracy, no false-positive rates, no confusion matrices, no plots. The abstract states that 'the ML approach outperforms static thresholds in accuracy,' but the evaluation section is essentially a summary. A claim without data is not a result.\n\nEven if the numbers were there, the experiment cannot support the claim. In Algorithm 2, the timing feature t_i is recorded after the injected sleep, so the artificial delay is part of the input to both the threshold and the classifier. Similarly, the ML features include the original block bytes, which for fault-injected blocks contain the XOR-modified first byte. So both detectors are keyed on the injection markers themselves, not on any genuine side-channel leakage. The comparison is therefore between two ways of detecting the injection, not between two ways of detecting a side-channel attack. On top of that, the threshold is computed and evaluated on the same dataset (in-sample), and the anomaly ratio of 10–30% is high enough to make the classification task trivial.\n\nTo be fair, the paper does not misrepresent the anomalies as real attacks; it calls them simulations. But the abstract and introduction make the broader claim of 'side-channel attack detection,' and that framing is not supported. Real timing leaks are often nanoseconds, and real fault attacks corrupt intermediate state, not plaintext bytes. The detection accuracy reported here would not transfer.\n\nThis is early-stage work, maybe an undergraduate project. A reader looking for an example of how to set up AES monitoring on a PYNQ might find the algorithm sketches useful. But it is not a security result. I would not send it to peer review in its current form. If the authors strip the injection markers from the features, validate against real attack traces, and report standard metrics on a held-out set, it could become a modest engineering note.","headline":"No results are reported, and the detector features include the injected delays and bit flips, so the claimed ML-vs-threshold comparison cannot support any conclusion about side-channel detection.","tokens_in":4377,"tokens_out":2720,"would_cite":false,"duration_ms":31347,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"ML beats timing thresholds for AES side-channel attacks on embedded hardware.","keywords":["AES-128","side-channel attack","timing attack","fault injection","anomaly detection","Random Forest","FPGA","PYNQ"],"falsifier":"Run the detector on real traces: mount a cache-timing attack on an AES-128 implementation with microsecond-scale timing variations, or inject a fault into an intermediate AES round, then compare detection accuracy against the reported simulation numbers. If accuracy collapses to chance, the central claim that the framework detects real side-channel attacks fails.","tokens_in":3545,"feed_emoji":"🛡️","tokens_out":6456,"duration_ms":64255,"temperature":0.7,"pith_summary":"This paper sets out to show that a lightweight, two-part detector can catch side-channel-style anomalies around AES-128 encryption on a small embedded system, without altering the cipher or using hardware performance counters. The first part is a statistical threshold on execution time; the second is a Random Forest classifier that learns from timing and ciphertext bytes after each block. The authors inject two kinds of synthetic attacks—a 5–20 ms delay before encryption and a bit flip in the first plaintext byte—and measure how well each detector flags the tampered blocks. Their claimed result is that the Random Forest beats the static threshold on accuracy while staying real-time: under 5 ms inference per block on a PYNQ-Z1 and using under 30% of the FPGA fabric. A sympathetic reader would take this as evidence that cheap, non-invasive machine-learning monitoring can harden embedded cryptography against timing and fault side-channels.","feed_headline":"ML beats timing thresholds for AES side-channel attacks","feed_subtitle":"Embedded PYNQ-Z1 runs the Random Forest detector in under 5 ms per block with under 30% FPGA resources.","key_machinery":"Two detection mechanisms carry the argument. (1) A statistical threshold, T = μ + 3·(max−min)/n, flags a block as malicious when its AES execution time exceeds the mean plus three times the normalized range; this is the fast, low-overhead baseline. (2) A Random Forest classifier, fed the per-block encryption time and the full 16-byte ciphertext, learns the boundary between normal and anomaly-injected blocks; this is the higher-accuracy detector. The anomaly injection module—a 5–20 ms sleep before encryption for timing attacks and an XOR of the first plaintext byte with 0xFF for fault attacks—provides the labeled training signal and the ground truth for evaluation.","core_discovery":"The paper's central claim is that a hybrid monitor combining a simple timing threshold with a Random Forest classifier can detect simulated timing and fault anomalies in AES-128 encryption blocks with high accuracy on both a desktop CPU and a PYNQ-Z1 SoC. The threshold method flags any block whose encryption time exceeds T = μ + 3·(max−min)/n, a heuristic analog of the 3-sigma rule. The Random Forest is trained on per-block timing plus the 16 ciphertext bytes, with block labels derived from the simulated injections. The authors report that the ML approach outperforms the static threshold on accuracy, returns predictions in under 5 ms on the embedded ARM Cortex-A9, and consumes less than 30%","pith_inferences":["The simulated anomalies are orders of magnitude larger than real side-channel leaks: real timing attacks exploit cache misses or variable-time operations at microsecond/nanosecond scale, and real fault injection usually targets intermediate AES state, not the first plaintext byte. If those real attacks are the intended threat, the reported accuracy should be expected to drop unless the feature set","The detector's block-level labels are known only because the paper controls anomaly injection. In a live system, anomalies do not come with labels, so the Random Forest would need an unsupervised or one-class variant to detect novel attacks without retraining on each new attack type.","A testable extension: sweep the injected delay from 20 ms down to microseconds to map how detection accuracy degrades; the crossover point would indicate the smallest timing anomaly this feature set can catch."],"forward_implications":["On embedded platforms like the PYNQ-Z1, a Random Forest detector can flag anomalous AES blocks in real time (under 5 ms per block) using only Python-level timing and ciphertext, without hardware counter support.","The threshold method remains useful as a zero-training, low-overhead first pass; the ML layer can be reserved for blocks the threshold flags or for finer discrimination.","Since the framework stays outside the AES core, it can be retrofitted to existing cryptographic libraries and does not require constant-time rewrites or masking.","The reported resource footprint (under 30% FPGA utilization) suggests the detector can coexist with the encryption engine and other peripherals on small SoCs.","Detection accuracy is bounded by the richness of the features—ciphertext bytes plus one timing value per block—so the approach is deliberately feature-light and transferable across platforms, with consistent results on CPU and FPGA."],"supporting_citations":[{"why":"Establishes that implementation timing leaks secret information, motivating the timing side-channel detection target.","marker":"[2]"},{"why":"Shows cache-timing attacks on AES can be mounted remotely, framing the need for the detector.","marker":"[3]"},{"why":"Introduces differential fault analysis, motivating the fault-injection anomaly scenario.","marker":"[4]"},{"why":"Prior work on machine-learning side-channel detection for FPGA SoCs; the paper positions its approach relative to this.","marker":"[10]"},{"why":"Supplies the PYNQ-Z1 SoC platform used for the embedded evaluation.","marker":"[12]"},{"why":"Provides the AES-128 ECB encryption implementation used in all experiments.","marker":"[13]"},{"why":"Source of the empirical 3-sigma-style rule that the statistical threshold is based on.","marker":"[14]"}],"fun_headline_variants":["Random Forest detects AES side-channel attacks on PYNQ","ML outshines timing thresholds for AES on embedded SoCs","Hybrid ML monitor thwarts AES side-channel attacks in under 5ms","Lightweight AI detects AES attacks on low-power SoCs","AES defense: Random Forest beats static timing checks"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The injected anomalies—a 5–20 ms delay and a first-byte XOR—are assumed to be faithful proxies for real timing and fault side-channel attacks; if real attacks are far smaller or corrupt different parts of the computation, the measured accuracy will not carry over.","fun_headline_variants_meta":{"raw":{"variants":["Random Forest detects AES side-channel attacks on PYNQ","ML outshines timing thresholds for AES on embedded SoCs","Hybrid ML monitor thwarts AES side-channel attacks in under 5ms","Lightweight AI detects AES attacks on low-power SoCs","AES defense: Random Forest beats static timing checks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000263,"raw_usage":{"total_tokens":1404,"prompt_tokens":681,"completion_tokens":723,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":425,"completion_tokens_details":{"reasoning_tokens":638}},"tokens_in":425,"tokens_out":723,"duration_ms":8216,"temperature":1.0,"reasoning_tokens":638,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T14:07:26.423050+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the detector on real traces: mount a cache-timing attack on an AES-128 implementation with microsecond-scale timing variations, or inject a fault into an intermediate AES round, then compare detection accuracy against the reported simulation numbers. If accuracy collapses to chance, the central claim that the framework detects real side-channel attacks fails.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes that implementation timing leaks secret information, motivating the timing side-channel detection target."},{"cited_title":"Bernstein","cited_arxiv_id":null,"evidence_quote":"Shows cache-timing attacks on AES can be mounted remotely, framing the need for the detector."},{"cited_title":"Differential fault analysis of secret key cryptosystems","cited_arxiv_id":null,"evidence_quote":"Introduces differential fault analysis, motivating the fault-injection anomaly scenario."},{"cited_title":"Machine- learning-based side-channel attack detection for fpga socs","cited_arxiv_id":null,"evidence_quote":"Prior work on machine-learning side-channel detection for FPGA SoCs; the paper positions its approach relative to this."},{"cited_title":"Pynq-z1: Python productivity for zynq","cited_arxiv_id":null,"evidence_quote":"Supplies the PYNQ-Z1 SoC platform used for the embedded evaluation."},{"cited_title":"Pycryptodome: Python cryptographic library","cited_arxiv_id":null,"evidence_quote":"Provides the AES-128 ECB encryption implementation used in all experiments."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Source of the empirical 3-sigma-style rule that the statistical threshold is based on."}],"review_version":1}