{"id":"54de6ada-13ef-4e76-af3c-5de8704b807e","arxiv_id":"2504.18990","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"In simulation, AEB with independent sensors and human driver intervention outperform a simple ML mitigation baseline at preventing OpenPilot crashes caused by ACC and lane-centering perception attacks.","lead":"This paper tests how well existing safety systems, like automatic emergency braking and human drivers, prevent crashes after an attack on a driver-assistance system's camera inputs. Using a driving simulator with the open-source OpenPilot software, it finds that independent-sensor emergency braking and alert drivers prevent most simulated collisions, but lateral lane-drift attacks remain hard to stop.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Fault-injection magnitudes are assumed to match physical adversarial patches; the paper never validates this mapping, so Table VI may measure a different phenomenon.","rationale":"The reader identified the same load-bearing concern as I did: the equivalence between fault injection and physical adversarial patches is assumed without validation. This concern directly threatens the central claim because the quantitative prevention rates in Table VI are the evidence for Observation 3, and those rates are produced under injected perturbations abstracted from prior work rather than from patches actually applied to OpenPilot v0.9.7. The paper is otherwise transparent, internally consistent, and releases code, which supports a conditional verdict rather than rejection. Since the reader already issued a CONDITIONAL verdict based on this exact concern, my analysis does not move the verdict; it reinforces the need for the stated condition: the fault-injection-to-patch mapping must be validated or the conclusions re-scoped. No additional load-bearing issues were found: the driver reaction model is varied experimentally, the ML baseline is explicitly described as a basic comparison, and the simulation-only scope is acknowledged in the Threats to Validity.","tokens_in":15664,"tokens_out":3935,"duration_ms":38107,"concrete_test":"","verdict_should_be":"UNCHANGED","load_bearing_attack":"All central numerical claims (e.g., AEB with independent sensor prevents 100% of relative-distance accidents, driver interventions prevent 40%–69.17%, ML baseline 23.08%–40%) rest on the assumption that the source-level fault-injection parameters in Table III faithfully represent physical adversarial-patch effects on OpenPilot v0.9.7's perception. Section III-B states: 'we directly emulate the effect of the patches by injecting attacks into the DNN output and getting the range of attack values of mispredictions corresponding to adversarial patches from previous work [9], [10].' This is a transfer assumption across systems and versions: [9] and [10] target different ADAS versions or surrogate models, and OpenPilot v0.9.7's DNN may have different sensitivity, output distributions, or failure modes. If real patches induce errors that differ in magnitude, timing, or trigger envelope (e.g., the lead-vehicle patch is detected at different distances, or causes a complete misdetection rather than a bounded distance offset), then every prevention rate in Table VI, and hence Observation 3, quantifies a different phenomenon than 'adversarial patch attacks.' The paper's Threats to Validity acknowledge simulation-only scope but do not validate the injection-to-patch mapping, so the central claim is not yet empirically tied to the attack vector named in the title.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper evaluates the resilience of OpenPilot v0.9.7, an open-source Level-2 ADAS, against adversarial perception attacks by emulating attack effects through source-level fault injection into DNN outputs (relative-distance errors and curvature deviations borrowed from prior work [9], [10]). It integrates OpenPilot with the MetaDrive simulator and implements safety mechanisms—AEBS in three configurations, firmware-style safety checks, a rule-based driver reaction simulator, and an LSTM-based ML mitigation baseline—then measures accident prevention rates across six driving scenarios. The authors report that OpenPilot is vulnerable to the emulated attacks, that AEB with independent sensor data and timely human intervention prevent a large fraction of accidents (up to 100% for relative-distance attacks), that lateral-direction attacks remain difficult to mitigate, and that the ML baseline underperforms basic safety mechanisms. They also identify conflicts among safety interventions, particularly the priority of AEB overriding driver inputs, as an important design issue.","tokens_in":15933,"tokens_out":4203,"duration_ms":42070,"significance":"If the results are accepted, the paper makes a useful contribution by building an open-source, closed-loop simulation platform that couples real ADAS control software (OpenPilot) with a physical-world simulator and multiple safety mechanisms; the artifact is available at a DOI, which aids reproducibility. The comparison of AEBS configurations, human interventions, and an ML-based mitigation method is a valuable empirical evaluation for the dependable-systems and automotive-security communities. The observation that independent-sensor AEB outperforms both ML mitigation and delayed driver reactions is plausible and practically relevant. However, the significance is limited by the unvalidated assumption that the fixed fault-injection magnitudes faithfully represent physical adversarial-patch effects, and by the small number of repetitions per configuration. The paper's main value is as a fault-injection-based sensitivity study of safety interventions, not as a direct evaluation of physical adversarial patches as the title suggests.","major_comments":[{"comment":"The central empirical claim rests on an unvalidated transfer assumption. The paper emulates adversarial patches by injecting fixed perception errors (10–38 m relative-distance errors and a 3% curvature deviation) taken from prior work [9], [10], but it does not demonstrate that these magnitudes, trigger conditions, and error shapes reproduce what an actual adversarial patch does to OpenPilot v0.9.7's DNN inside MetaDrive. Different model versions, camera parameters, or driving environments could produce different error profiles, so every prevention rate in Table VI, and hence Observation 3, may quantify a different phenomenon than 'adversarial patch attack.' I recommend either validating the mapping with at least one real-patch evaluation on this platform or reframing the claims as an evaluation of safety interventions under fault-injection-emulated perception errors, and adjusting the title and abstract accordingly.","section":"§III-B / Table VI"},{"comment":"Each fault-injection configuration is repeated only 10 times, but results are presented as point percentages with no variance or confidence intervals. With 10 trials, a single accident shifts a prevention rate by 10 percentage points, which makes the main comparative claims (e.g., 40% driver braking vs. 100% AEB for relative-distance attacks, and the non-monotonic reaction-time results in Table VII) statistically fragile. Please report per-run outcomes, confidence intervals, or at least state the exact number of trials for every cell and discuss the resulting uncertainty when making comparative statements.","section":"§IV-E (Table VI, Table VII)"},{"comment":"The ML baseline's behavior depends on the threshold τ and the bias parameter b0, but the paper never reports their values or the procedure used to set them. Without this information, the comparison in Table VI that supports Observation 6 is not reproducible and could be sensitive to hyperparameter choices. Please report the exact settings or provide a sensitivity analysis over τ and b0 to demonstrate that the comparison with AEB and driver intervention is fair and robust.","section":"§IV-D / Algorithm 1"}],"minor_comments":[{"comment":"The title and abstract repeatedly say 'adversarial patches,' but Section III-B states that the work 'directly emulates' patch effects by injecting attacks into DNN outputs; please use terminology that consistently distinguishes emulated perception errors from physically realized adversarial patches.","section":"Title, Abstract, §III-B"},{"comment":"The text says each configuration is repeated 10 times, yet Table IV reports benign-condition outcomes with denominators of 20 (e.g., '1/20', '10/20'); please clarify how many runs were performed for the no-attack baseline and whether the 360-simulation count refers only to fault-injection runs.","section":"§IV / Table IV"},{"comment":"The Threats to Validity section mentions simulation-only scope and driver-model simplification, but it does not list the fault-injection-to-patch mapping as a limitation; this is a significant validity threat and should be acknowledged explicitly.","section":"§V"},{"comment":"Table VI is dense and the header abbreviations ('Comp.', 'Indep.', and the multiple 'Trigger Rate' columns) are hard to parse; consider splitting the table into sub-tables by fault type or adding a legend that explains each row's intervention configuration.","section":"Table VI"},{"comment":"The speed-dependent TTC thresholds (3.8, 5.8, 9.8) are introduced without explaining their units or provenance beyond a citation to prior work; a brief sentence clarifying their basis would improve reproducibility.","section":"§III-C, Eq. (4)"},{"comment":"The claim that the ML model 'prevented nearly all A1 accidents caused by relative distance attacks' is supported by the table, but it would benefit from a per-scenario breakdown given the small number of trials and the wide spread in the other rows.","section":"§IV-E6"}],"recommendation":"major_revision","confidential_remarks":"The paper's central artifact—an open-source platform combining OpenPilot, MetaDrive, and multiple safety mechanisms—is a legitimate contribution, and the comparison framework is well scoped. The main gap is that the title and abstract promise 'adversarial patches' while the experiments use fault-injection emulation with no validation of equivalence; this must be either closed or honestly reframed. The small number of repetitions (10 per configuration) also weakens several comparative claims. No concerns about novelty disclosure; the self-citation to [9] is appropriate given the platform lineage. I would support publication after a revision that addresses the validation gap and the statistical-reporting issues."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The one thing to know: this is a solid simulation study of how safety layers behave under perception faults, but it does not actually run adversarial patches. The authors inject fixed errors into OpenPilot's DNN outputs, with magnitudes taken from prior work on other systems and versions. All the prevention rates in Table VI are conditional on that unvalidated mapping. That said, the core system-level findings are useful and defensible as a scoped fault-injection study.\n\nWhat is new and what the paper does well: it integrates the real OpenPilot v0.9.7 control software with MetaDrive, implements a standards-based AEB with three data-source configurations, a rule-based driver reaction model, firmware safety checks, and a simple LSTM baseline. That is a genuinely useful testbed, and the code is released. The main qualitative results are interesting: AEB with an independent sensor prevents essentially all forward collisions under the tested injections, driver braking and steering matter a lot, the ML baseline is clearly worse, and there is a real arbitration conflict where AEB's priority can undermine driver steering in lateral scenarios. Those observations have value for ADAS safety engineering and for anyone thinking about intervention priority.\n\nThe soft spots, in proportion: the injection-to-patch transfer assumption is load-bearing. The paper's title and abstract claim evaluation against adversarial patches, but no patch is generated or evaluated; the attack values come from earlier papers that targeted different systems or surrogate models. There is no evidence that a real patch on OpenPilot's DNN produces the same magnitude, timing, or trigger envelope. The authors acknowledge the simulation-only scope in the threats section, but they do not validate the mapping itself. That should be stated plainly as a limitation, or the paper should be reframed as a fault-injection robustness study. Less serious but still worth fixing: only 10 runs per configuration, with results reported as point percentages and no variance or confidence intervals. With 10 runs, '100%' means 0 accidents in 10, which is not the same statistical evidence as a larger sample. The ML baseline is deliberately simple, so the claim that basic mechanisms outperform 'certain ML-based mitigation methods' is narrow and should not be generalized.\n\nCitation pattern is fine; the self-citations are relevant prior work that the study builds on, not padding. The central qualitative argument holds up as a scoped simulation claim: safety layers, especially independent-sensor AEB and alert drivers, can prevent a large fraction of accidents from moderate perception errors, and arbitration matters. The quantitative rates should be treated as illustrative, not predictive.\n\nWho is this for? Readers working on ADAS safety, dependable autonomy, and physical adversarial machine learning. It deserves a serious referee. I would recommend major revision: either validate the fault-to-patch mapping (at least with a small study against a surrogate patch model) or honestly reframe the contribution as fault-injection-based resilience analysis; add variance reporting; and tighten the ML baseline comparison language.","headline":"A useful, honestly scoped simulation study of safety interventions against perception faults in OpenPilot, but the title overstates the attack model—the paper injects fixed DNN-output errors from prior work rather than evaluating real adversarial patches.","tokens_in":16487,"tokens_out":1644,"would_cite":true,"duration_ms":20255,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Auto-braking and drivers can beat ML fixes for camera patches.","keywords":["adversarial patches","ADAS","automatic emergency braking","fault injection","driver intervention","OpenPilot","perception attacks","safety intervention coordination"],"falsifier":"Place a real printed or projected adversarial patch on a lead vehicle and on the road ahead of a car running the same assisted-driving software, instrument the perception module to record the relative-distance and curvature errors actually produced, and check whether those errors fall in the injected ranges at the assumed trigger distances; if they do not, the reported prevention rates and mechanism rankings need to be recomputed.","tokens_in":15473,"feed_emoji":"🚗","tokens_out":9987,"duration_ms":85912,"temperature":0.7,"pith_summary":"The paper asks whether the safety machinery already built into a modern Level-2 driver assistance system—automatic emergency braking, forward-collision warnings, command-range safety checks, and the human driver—can absorb the effects of adversarial patches that fool the camera, and whether an added machine-learning safeguard would do better. By simulating such a system under attacks that corrupt relative-distance and lane-curvature predictions, it finds that existing mechanisms are the main line of defense: AEB fed by an independent sensor prevents up to 100% of simulated forward collisions, driver reactions prevent 40–69% of accidents depending on attack type, and the ML baseline prevents only 23–40%. It also finds that priority conflicts can erode safety, as when high-priority AEB overrides a driver's better-suited response to lateral attacks. If this is right, the resilience discussion shifts from adding new ML defenses to hardening the sensors behind AEB and coordinating interventions.","feed_headline":"Auto-braking and drivers can beat ML fixes for camera patches","feed_subtitle":"Simulated attacks on OpenPilot find AEB with independent sensors prevents up to 100% of crashes.","key_machinery":"The load-bearing mechanism is a time-to-collision-based emergency-braking controller, with $ttc = RD/RS$ (relative distance divided by relative speed), that issues a forward-collision warning and then applies staged braking (90%, 95%, then full force) as TTC falls below speed-dependent thresholds. The paper varies one property of that mechanism: whether AEB receives the same corrupted perception predictions that drive the controller, no sensor input at all, or an independent secure sensor stream. Around this controller sit a command-range safety checker that blocks gas and brake commands outside 2 to -3.5 m/s², and a driver-reaction simulator with fixed reaction times that brakes or steers back toward lane center when warnings trigger. The fault-injection engine that emulates adversarial patches by directly perturbing relative distance (10–38 m) and desired curvature (3%) is what connects this safety stack to physical attacks. The argument works by comparing accident-prevention rates across configurations of this stack.","core_discovery":"On the paper's own terms, the central discovery is that a real Level-2 ADAS is not defenseless against perception attacks: the defense is already in the car. OpenPilot alone collides or leaves the lane in every simulated attack configuration, but adding a time-to-collision-based AEB that reads an independent, uncompromised data stream prevents 100% of forward-collision accidents from relative-distance attacks, whereas the same AEB fed by the same corrupted perception stream prevents only 19.17%. Human drivers reacting after 2.5 seconds prevent 40–69.17% of accidents depending on attack type, and an LSTM (long short-term memory) machine-learning mitigation model prevents 23.08–40%. Lateral lane-centering attacks remain only partially mitigated—AEB's indirect prevention of lateral accidents tops out near 40.83%—and in mixed attacks the fixed priority order that lets AEB override the driver lowers the combined prevention rate from 69.17% to about 51.67%. These numbers are the paper's claim, not a field-wide result.","pith_inferences":["Editorial inference: the 100% figure for independent-sensor AEB suggests that adding a redundant perception channel—radar or a second camera with different failure modes—is likely to buy more real-world safety than retraining the control model, but only if the redundant channel actually covers the same failure modes as the attacked camera.","Editorial inference: because the fault-injection magnitudes come from prior work on different perception models, the exact percentages should be read as ordering evidence rather than predictive field rates; a real patch that induces a different error distribution, or errors that vary over time, could reorder the mechanisms.","Editorial inference: a testable extension would run the same scenario suite with adaptive or temporally varying fault magnitudes to see whether the rank ordering survives; the fixed 10–38 m and 3% magnitudes are the point where the simulation is most likely to diverge from physics.","Editorial inference: the finding that AEB prevents some lateral accidents implies that longitudinal emergency braking doubles as a coarse lateral safety net; designers could exploit this deliberately by coupling AEB activation to lane-departure risk rather than treating the two directions as independent."],"forward_implications":["If AEB reads an independent sensor, forward-collision prevention reaches 100% in the simulated relative-distance attacks; the same AEB on compromised data prevents only 19.17%, so redundant sensing is the single highest-leverage safety feature tested.","Driver reaction time matters measurably: shortening it from 2.5 s to 1.0 s raises accident prevention on relative-distance attacks from 40% to 53.33%, and on curvature attacks from 48.33% to 77.50%.","Lateral attacks are the hard case: no tested mechanism exceeds 53.33% prevention for curvature attacks, and AEB only helps because aggressive acceleration toward the lead vehicle triggers braking before the ego vehicle leaves the lane.","Priority conflicts degrade safety: when AEB overrides the driver in mixed attacks, the combined prevention rate falls from 69.17% (driver alone) to 50–51.67%, so intervention coordination is itself a safety parameter.","An ML recovery model trained on fault-free commands is not a substitute for these mechanisms: it prevents 23.08–40% of accidents and even introduces new lateral accidents in relative-distance attacks."],"supporting_citations":[{"why":"Supplies the relative-distance attack magnitudes (10–38 m) and the TTC-based AEB design.","marker":"[9]"},{"why":"Supplies the lane-centering attack mechanism and the 3% curvature deviation magnitude.","marker":"[10]"},{"why":"Provides the open-source driver assistance system under test.","marker":"[22]"},{"why":"Provides the driving simulator used for all closed-loop experiments.","marker":"[24]"},{"why":"Supports the threat model of a physical patch placed on the lead vehicle's rear.","marker":"[26]"},{"why":"Grounds the AEBS implementation in real regulation-based braking phases.","marker":"[31]"},{"why":"Supplies the multi-stage emergency braking algorithm adopted for the AEB model.","marker":"[34]"},{"why":"Supplies the driver emergency-braking behavior model used by the reaction simulator.","marker":"[40]"},{"why":"Provides the sensor-isolation recovery design that the ML mitigation baseline is patterned on.","marker":"[20]"}],"fun_headline_variants":["Independent AEB prevents all simulated forward crashes","Human drivers beat ML models against patch attacks","OpenPilot survives patches with independent-sensor AEB","AEB with separate sensors blocks 100% of forward collisions","Driver reaction outperforms LSTM vs adversarial patches"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper's prevention rates all stand on the assumption that injecting fixed errors into the perception output—10–38 m in relative distance and a 3% curvature deviation—faithfully reproduces what a physical adversarial patch does to a camera-based assisted-driving system in this simulator; if that mapping is off, every comparison in the paper measures a different attack than the one claimed.","fun_headline_variants_meta":{"raw":{"variants":["Independent AEB prevents all simulated forward crashes","Human drivers beat ML models against patch attacks","OpenPilot survives patches with independent-sensor AEB","AEB with separate sensors blocks 100% of forward collisions","Driver reaction outperforms LSTM vs adversarial patches"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000794,"raw_usage":{"total_tokens":3463,"prompt_tokens":877,"completion_tokens":2586,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":493,"completion_tokens_details":{"reasoning_tokens":2512}},"tokens_in":493,"tokens_out":2586,"duration_ms":18912,"temperature":1.0,"reasoning_tokens":2512,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T10:04:03.863064+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Place a real printed or projected adversarial patch on a lead vehicle and on the road ahead of a car running the same assisted-driving software, instrument the perception module to record the relative-distance and curvature errors actually produced, and check whether those errors fall in the injected ranges at the assumed trigger distances; if they do not, the reported prevention rates and mechanism rankings need to be recomputed.","supporting_citations":[{"cited_title":"Runtime Stealthy Perception Attacks against DNN- Based Adaptive Cruise Control Systems,","cited_arxiv_id":null,"evidence_quote":"Supplies the relative-distance attack magnitudes (10–38 m) and the TTC-based AEB design."},{"cited_title":"Dirty Road Can Attack: Security of Deep Learning Based Automated Lane Centering under{Physical-World}Attack,","cited_arxiv_id":null,"evidence_quote":"Supplies the lane-centering attack mechanism and the 3% curvature deviation magnitude."},{"cited_title":"OpenPilot","cited_arxiv_id":null,"evidence_quote":"Provides the open-source driver assistance system under test."},{"cited_title":"Implementation of Autonomous Emergency Braking (AEB), the Next Step in Euro NCAP’S Safety Assessment,","cited_arxiv_id":null,"evidence_quote":"Grounds the AEBS implementation in real regulation-based braking phases."},{"cited_title":"Autonomous Vehicle with Emergency Braking Algorithm Based on Multi-Sensor Fusion and Super Twisting Speed Controller,","cited_arxiv_id":null,"evidence_quote":"Supplies the multi-stage emergency braking algorithm adopted for the AEB model."},{"cited_title":"Driver Brake Response to Sudden Unintended Acceleration while Parking,","cited_arxiv_id":null,"evidence_quote":"Supplies the driver emergency-braking behavior model used by the reaction simulator."},{"cited_title":"PID- Piper: Recovering Robotic Vehicles from Physical Attacks,","cited_arxiv_id":null,"evidence_quote":"Provides the sensor-isolation recovery design that the ML mitigation baseline is patterned on."}],"review_version":1}