pith. sign in

arxiv: 2604.20895 · v1 · submitted 2026-04-21 · 💻 cs.CR · cs.CY· cs.LG

Towards a Systematic Risk Assessment of Deep Neural Network Limitations in Autonomous Driving Perception

Svetlana Pavlitska , Christopher Gerking , J. Marius Z\"ollner This is my paper

Pith reviewed 2026-05-10 02:51 UTC · model grok-4.3

classification 💻 cs.CR cs.CYcs.LG
keywords autonomous drivingdeep neural networksrisk assessmentISO 26262ISO/SAE 21434perception systemssafetysecurity
0
0 comments X

The pith

A joint workflow combining hazard and threat analyses from two ISO standards can systematically identify risks from deep neural network limitations in autonomous driving perception.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper proposes merging the hazard analysis and risk assessment process from ISO 26262 with the threat analysis and risk assessment from ISO/SAE 21434. This creates a single workflow to spot and evaluate dangers caused by shortcomings in deep neural networks used for perception in autonomous vehicles. Shortcomings addressed include lack of generalization to new situations, inefficiency, poor explainability, implausible outputs, and lack of robustness. A reader would care because autonomous driving systems rely heavily on these networks, and unexamined limitations could lead to safety failures or security breaches without a structured way to catch them.

Core claim

The authors claim that combining HARA following ISO 26262 and TARA following ISO/SAE 21434 produces a joint workflow that identifies and analyzes risks arising from inherent DNN limitations in autonomous driving perception.

What carries the argument

The joint HARA-TARA workflow, which maps DNN limitations such as lack of generalization, efficiency, explainability, plausibility, and robustness onto hazard and threat categories from the two standards.

If this is right

  • Risks from DNN limitations in perception can be categorized using established safety and security standards instead of ad-hoc methods.
  • The workflow makes it possible to trace how specific DNN shortcomings translate into concrete hazards or threats for autonomous vehicles.
  • Safety and security analyses become integrated, reducing the likelihood that interactions between them are overlooked during system development.
  • Early-stage risk assessment for autonomous driving stacks becomes feasible by applying the workflow to perception components.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The workflow could be tested on public autonomous driving datasets to check whether it surfaces limitations that have caused real incidents.
  • Similar joint analysis might apply to other parts of the autonomous driving stack, such as planning or control modules.
  • The mapping process may highlight cases where DNN-specific risks fall outside current ISO categories and require extensions.

Load-bearing premise

DNN limitations can be directly mapped onto the hazard and threat categories in the ISO standards without missing key interactions or needing entirely new metrics.

What would settle it

Applying the proposed workflow to a real autonomous driving perception DNN and discovering that it overlooks a documented failure mode, such as misperception under specific weather conditions or adversarial inputs, would show the mapping is incomplete.

Figures

Figures reproduced from arXiv: 2604.20895 by Christopher Gerking, J. Marius Z\"ollner, Svetlana Pavlitska.

Figure 1
Figure 1. Figure 1: Proposed combined HARA-TARA risk assessment. 4 Evaluation We apply the proposed workflow to each DNN limitation, derive hazards/threats, and assign the Automotive Safety Integrity Level (ASIL)/risk levels. Step 1: Item Definition: At the first stage, the items common for HARA and TARA are defined. An item in HARA is "a system . . . or combination of systems . . . , to which ISO 26262 is applied, that imple… view at source ↗
read the original abstract

Safety and security are essential for the admission and acceptance of automated and autonomous vehicles. Deep neural networks (DNNs) are widely used for perception and further components of the autonomous driving (AD) stack. However, they possess several limitations, including lack of generalization, efficiency, explainability, plausibility, and robustness. These insufficiencies can pose significant risks to autonomous driving systems. However, hazards, threats, and risks associated with DNN limitations in this domain have not been systematically studied so far. In this work, we propose a joint workflow for risk assessment combining the hazard analysis and risk assessment (HARA) following ISO 26262 and threat analysis and risk assessment (TARA) following the ISO/SAE 21434 to identify and analyze risks arising from inherent DNN limitations in AD perception.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript proposes a joint workflow for risk assessment in autonomous driving perception systems that integrates hazard analysis and risk assessment (HARA) as per ISO 26262 with threat analysis and risk assessment (TARA) as per ISO/SAE 21434. The goal is to systematically identify and analyze risks stemming from inherent limitations of deep neural networks, including lack of generalization, efficiency, explainability, plausibility, and robustness.

Significance. If the proposed workflow proves effective, it would offer a structured method to address both safety and security concerns associated with DNN limitations in AD systems, which is critical for their safe deployment and regulatory approval. The work highlights an important gap in current standards application to AI components. However, its significance is currently limited by the absence of concrete demonstrations.

major comments (2)
  1. The central assumption that DNN limitations can be slotted into the existing HARA and TARA categories without new metrics or extensions is not substantiated. For instance, the lack of explainability and plausibility in DNNs has no direct equivalent in the functional failure modes of ISO 26262 or the threat scenarios of ISO/SAE 21434. A detailed mapping table or example would be necessary to support the claim that the combined workflow can identify all relevant risks.
  2. The paper presents only a high-level conceptual proposal without any case study, validation data, or worked example of applying the workflow to a specific DNN limitation in AD perception. This makes it difficult to assess whether the approach can actually analyze the risks as claimed.
minor comments (2)
  1. The abstract could be more concise as it repeats the list of DNN limitations and the proposal description.
  2. Ensure that all references to ISO standards include the specific parts or clauses relevant to HARA and TARA for clarity.

Simulated Author's Rebuttal

2 responses · 0 unresolved

Thank you for the constructive feedback on our manuscript proposing a joint HARA-TARA workflow to systematically identify risks from inherent DNN limitations in autonomous driving perception. We address each major comment below, indicating revisions where appropriate to strengthen the work without misrepresenting its conceptual scope.

read point-by-point responses

  1. Referee: The central assumption that DNN limitations can be slotted into the existing HARA and TARA categories without new metrics or extensions is not substantiated. For instance, the lack of explainability and plausibility in DNNs has no direct equivalent in the functional failure modes of ISO 26262 or the threat scenarios of ISO/SAE 21434. A detailed mapping table or example would be necessary to support the claim that the combined workflow can identify all relevant risks.

    Authors: We agree that an explicit mapping is required to substantiate how the workflow integrates DNN limitations into the existing standards. The manuscript argues that these limitations manifest as hazards or threats within the frameworks (e.g., lack of robustness as a functional safety issue under HARA or as an attack surface under TARA), but we acknowledge the need for clarification. In the revised version, we will add a detailed mapping table aligning each limitation (generalization, efficiency, explainability, plausibility, robustness) to specific HARA hazard categories and TARA threat scenarios, with AD perception examples. This will show the slotting process while noting that the combined workflow involves targeted adaptations rather than entirely new metrics. revision: yes

  2. Referee: The paper presents only a high-level conceptual proposal without any case study, validation data, or worked example of applying the workflow to a specific DNN limitation in AD perception. This makes it difficult to assess whether the approach can actually analyze the risks as claimed.

    Authors: The manuscript is positioned as a conceptual proposal to address the gap in applying existing standards to AI components, consistent with its title 'Towards a Systematic Risk Assessment...'. We recognize that a worked example would improve evaluability. In the revision, we will incorporate a concise worked example applying the workflow to one limitation (e.g., lack of robustness in object detection for a highway scenario), detailing the HARA/TARA steps for risk identification and analysis. Full empirical validation or multi-scenario case studies with data would extend beyond the current scope and are noted as directions for future research. revision: partial

Circularity Check

0 steps flagged

No circularity in methodological proposal

full rationale

The paper proposes a joint workflow combining HARA (ISO 26262) and TARA (ISO/SAE 21434) to assess risks from DNN limitations in AD perception. It contains no equations, no fitted parameters, no predictions, and no derivation chain. The central claim is a methodological suggestion that references external standards without reducing to self-citation, self-definition, or input-by-construction. No load-bearing self-referential steps exist; the work is self-contained as a proposal.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The proposal rests on the assumption that existing ISO standards can be extended to cover DNN-specific limitations without new axioms or entities; no free parameters or invented entities are introduced.

axioms (2)
  • domain assumption ISO 26262 HARA process remains applicable when the item under analysis is a DNN-based perception module
    The paper invokes the standard directly for hazard identification without demonstrating that DNN failure modes fit the standard's item definition.
  • domain assumption ISO/SAE 21434 TARA process can be merged with HARA without loss of coverage for perception-related threats
    Compatibility of the two standards for this use case is assumed rather than derived.

pith-pipeline@v0.9.0 · 5447 in / 1379 out tokens · 45693 ms · 2026-05-10T02:51:08.984819+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

34 extracted references · 34 canonical work pages

  1. [1]

    EU AI Act: first regulation on artificial intelligence (2024)

    work page 2024

  2. [2]

    In: Conference on Computer Vision and Pattern Recognition (CVPR) - Workshops (2021)

    Adilova, L., Schulz, E., Akila, M., Houben, S., Schneider, J.D., Hüger, F., Wirtz, T.: Plants don’t walk on the street: Common-sense reasoning for reliable seman- tic segmentation. In: Conference on Computer Vision and Pattern Recognition (CVPR) - Workshops (2021)

    work page 2021

  3. [3]

    In: International Symposium on Software Reliability Engineering, ISSRE (2013)

    Beckers, K., Heisel, M., Frese, T., Hatebur, D.: A structured and model-based haz- ard analysis and risk assessment method for automotive systems. In: International Symposium on Software Reliability Engineering, ISSRE (2013)

    work page 2013

  4. [4]

    In: Conference on Computer Vision and Pattern Recognition (CVPR) - Workshops (2022)

    Bogdoll, D., Nitsche, M., Zöllner, J.M.: Anomaly detection in autonomous driving: A survey. In: Conference on Computer Vision and Pattern Recognition (CVPR) - Workshops (2022)

    work page 2022

  5. [5]

    In: Advances in Neural Information Processing Systems (NIPS) - Workshops (2017)

    Brown, T.B., Mané, D., Roy, A., Abadi, M., Gilmer, J.: Adversarial Patch. In: Advances in Neural Information Processing Systems (NIPS) - Workshops (2017)

    work page 2017

  6. [6]

    IEEE Trans

    Chia,W.M.D.,Keoh,S.L.,Goh,C.,Johnson,C.W.:Riskassessmentmethodologies for autonomous driving: A survey. IEEE Trans. Intell. Transp. Syst. (2022)

    work page 2022

  7. [7]

    In: Vehicular Technology Conference VTC (2021)

    Chia, W.M.D., Keoh, S.L., Michala, A.L., Goh, C.: Real-time recursive risk assessment framework for autonomous vehicle operations. In: Vehicular Technology Conference VTC (2021)

    work page 2021

  8. [8]

    In: Conference on Computer Vision and Pattern Recognition (CVPR) (2018)

    Eykholt, K., Evtimov, I., Fernandes, E., Li, B., Rahmati, A., Xiao, C., Prakash, A., Kohno, T., Song, D.: Robust physical-world attacks on deep learning visual clas- sification. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2018)

    work page 2018

  9. [9]

    IEEE Access (2023)

    Ghosh, S., Zaboli, A., Hong, J., Kwon, J.: An integrated approach of threat analysis for autonomous vehicles perception system. IEEE Access (2023)

    work page 2023

  10. [10]

    In: International Conference on Learning Representations (ICLR) (2015)

    Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and Harnessing Adversar- ial Examples. In: International Conference on Learning Representations (ICLR) (2015)

    work page 2015

  11. [11]

    Transportation Research Part C: Emerging Technologies (2024)

    Grosse, K., Alahi, A.: A qualitative ai security risk assessment of autonomous vehicles. Transportation Research Part C: Emerging Technologies (2024)

    work page 2024

  12. [12]

    In: International Conference on Learning Representations (ICLR) (2016)

    Han, S., Mao, H., Dally, W.J.: Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. In: International Conference on Learning Representations (ICLR) (2016)

    work page 2016

  13. [13]

    In: Deep Neural Networks and Data for Automated Driving: Robustness, Uncertainty Quantification, and Insights Towards Safety

    Houben, S., Abrecht, S., Akila, M., Bär, A., Brockherde, F., Feifel, P., Fingscheidt, T., Gannamaneni, S.S., Ghobadi, S.E., Hammam, A., et al.: Inspect, understand, overcome: A survey of practical methods for ai safety. In: Deep Neural Networks and Data for Automated Driving: Robustness, Uncertainty Quantification, and Insights Towards Safety. Springer (2022)

    work page 2022

  14. [14]

    In: Advances in Neural Information Processing Systems (NIPS) (2019)

    Ilyas, A., Santurkar, S., Tsipras, D., Engstrom, L., Tran, B., Madry, A.: Adversarial examples are not bugs, they are features. In: Advances in Neural Information Processing Systems (NIPS) (2019)

    work page 2019

  15. [15]

    ISO International Organization for Standardization (2022)

    ISO-21448: Road vehicles – Safety of the intended functionality. ISO International Organization for Standardization (2022)

    work page 2022

  16. [16]

    ISO International Organization for Standardization (2018)

    ISO-26262: Road vehicles — functional safety. ISO International Organization for Standardization (2018)

    work page 2018

  17. [17]

    ISO International Organization for Standardization (2020) 10 S

    ISO/IEC-TR-24028: Information technology — artificial intelligence — overview of trustworthiness in artificial intelligence. ISO International Organization for Standardization (2020) 10 S. Pavlitska et al

    work page 2020

  18. [18]

    ISO International Organization for Standardization (2021)

    ISO/IEC-TR-24029: Artificial intelligence (ai) — assessment of the robustness of neural networks. ISO International Organization for Standardization (2021)

    work page 2021

  19. [19]

    ISO International Organization for Standardization (2024)

    ISO/IEC-TR-5469: Artificial intelligence - functional safety and ai systems. ISO International Organization for Standardization (2024)

    work page 2024

  20. [20]

    ISO Inter- national Organization for Standardization (2024)

    ISO/PAS8800:2024: Road vehicles — safety and artificial intelligence. ISO Inter- national Organization for Standardization (2024)

    work page 2024

  21. [21]

    ISO International Organization for Standardization (2021)

    ISO/SAE-21434: Road vehicles – cybersecurity engineering. ISO International Organization for Standardization (2021)

    work page 2021

  22. [22]

    ISO International Organization for Standardization (2022)

    ISO/SAE-PAS-22736: Taxonomy and definitions for terms related to driving automation systems for on-road motor vehicles. ISO International Organization for Standardization (2022)

    work page 2022

  23. [23]

    In: Proceedings of the Winter Conference on Applica- tions of Computer Vision (WACV) (2022)

    Nesti, F., Rossolini, G., Nair, S., Biondi, A., Buttazzo, G.C.: Evaluating the robustness of semantic segmentation for autonomous driving against real-world adversarial patch attacks. In: Proceedings of the Winter Conference on Applica- tions of Computer Vision (WACV) (2022)

    work page 2022

  24. [24]

    In: Intelligent Vehicles Symposium (IV) (2025)

    Pavlitska, S., Robb, J., Polley, N., Yazgan, M., Zöllner, J.M.: Fool the stoplight: Realistic adversarial patch attacks on traffic light detectors. In: Intelligent Vehicles Symposium (IV) (2025)

    work page 2025

  25. [25]

    In: International Conference on Intelligent Transportation Systems (ITSC) (2020)

    Pavlitskaya, S., Ünver, S., Zöllner, J.M.: Feasibility and suppression of adversarial patch attacks on end-to-end vehicle control. In: International Conference on Intelligent Transportation Systems (ITSC) (2020)

    work page 2020

  26. [26]

    Nature machine intelligence (2019)

    Rudin, C.: Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature machine intelligence (2019)

    work page 2019

  27. [27]

    CoRRabs/2002.08935(2020)

    Sämann, T., Schlicht, P., Hüger, F.: Strategy to increase the safety of a dnn-based perception for HAD systems. CoRRabs/2002.08935(2020)

    work page arXiv 2002

  28. [28]

    In: International Conference on Computer Safety, Reliability and Security (SafeComp) - Workshops (2020)

    Schwalbe, G., Knie, B., Sämann, T., Dobberphul, T., Gauerhof, L., Raafatnia, S., Rocco, V.: Structuring the safety argumentation for deep neural network based perception in automotive applications. In: International Conference on Computer Safety, Reliability and Security (SafeComp) - Workshops (2020)

    work page 2020

  29. [29]

    International Conference on Learning Representations (ICLR) (2014)

    Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., Fergus, R.: Intriguing Properties of Neural Networks. International Conference on Learning Representations (ICLR) (2014)

    work page 2014

  30. [30]

    Under- writers Laboratoriesn (2022)

    UL-4600: Standard for safety for the evaluation of autonomous products. Under- writers Laboratoriesn (2022)

    work page 2022

  31. [31]

    In: European Conference on Computer Vision (ECCV) - Workshops (2022)

    Vivekanandan, A., Maier, N., Zöllner, J.M.: Plausibility verification for 3d object detectors using energy-based optimization. In: European Conference on Computer Vision (ECCV) - Workshops (2022)

    work page 2022

  32. [32]

    In: International Conference on Dependable Systems and Networks - Workshops (2020)

    Warg, F., Skoglund, M.A., Thorsén, A., Johansson, R., Brännström, M., Gyllen- hammar, M., Sanfridson, M.: The quantitative risk norm - A proposed tailoring of HARA for ADS. In: International Conference on Dependable Systems and Networks - Workshops (2020)

    work page 2020

  33. [33]

    IEEE Transactions on Pattern Analysis and Machine Intelligence (2023)

    Wei, X., Guo, Y., Yu, J.: Adversarial sticker: A stealthy attack method in the physical world. IEEE Transactions on Pattern Analysis and Machine Intelligence (2023)

    work page 2023

  34. [34]

    In: International Conference on Computer Safety, Reliability and Security (SafeComp) - Workshops (2020)

    Willers, O., Sudholt, S., Raafatnia, S., Abrecht, S.: Safety concerns and mitigation approaches regarding the use of deep learning in safety-critical perception tasks. In: International Conference on Computer Safety, Reliability and Security (SafeComp) - Workshops (2020)

    work page 2020