Autonomous Systems Dependability in the era of AI: Design Challenges in Safety, Security, Reliability and Certification
Pith reviewed 2026-05-07 06:37 UTC · model grok-4.3
The pith
Traditional reliability, safety, and security methods cannot handle the uncertain behaviors of AI components in autonomous systems under real-time and power limits.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The design of embedded safety-critical systems is increasingly challenged by escalating system complexity, hardware-software heterogeneity, and the integration of intelligent, data-driven components. Ensuring dependability requires a holistic approach that spans multiple abstraction layers and encompasses both design- and run-time assurance. Traditional methods for reliability, safety, and security management often fall short in addressing the dynamic and uncertain behaviors introduced by AI and ML components, especially under stringent real-time, power, and safety constraints. While AI and ML offer powerful predictive, adaptive, and self-optimizing capabilities that can enhance system, the
What carries the argument
holistic multi-layer dependability framework that spans design-time and run-time assurance while accounting for imperfect, learning-enabled AI components
Load-bearing premise
Emerging methodologies, architectures, and frameworks can effectively account for imperfect, learning-enabled AI components to bridge the gap to certifiable system-level dependability.
What would settle it
A deployed autonomous system containing AI or ML components that achieves full certification for safety, security, and reliability using only traditional pre-AI methods without any new AI-aware modeling or runtime assurance techniques.
Figures
read the original abstract
The design of embedded safety-critical systems such as those used in next-generation automotive and autonomous platforms, is increasingly challenged by escalating system complexity, hardware-software heterogeneity, and the integration of intelligent, data-driven components. Ensuring dependability in such systems requires a holistic approach that spans multiple abstraction layers and encompasses both design- and run-time assurance. Traditional methods for reliability, safety, and security management often fall short in addressing the dynamic and uncertain behaviors introduced by Artificial Intelligence (AI) and Machine Learning (ML) components, especially under stringent real-time, power, and safety constraints. While AI and ML offer powerful predictive, adaptive, and self-optimizing capabilities that can enhance system dependability, their inherent non-determinism, data-dependence, and lack of formal guarantees introduce new challenges for verification, validation, and certification. This paper explores emerging methodologies, architectures, and frameworks for designing dependable autonomous and embedded systems in the era of AI. It highlight advances in reliability modeling, secure system design, and certification approaches that account for imperfect, learning-enabled components, aiming to bridge the gap between AI innovation and certifiable system-level dependability.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper argues that traditional methods for reliability, safety, and security management in embedded safety-critical systems (e.g., next-generation automotive and autonomous platforms) fall short when addressing the dynamic, uncertain, and non-deterministic behaviors of integrated AI/ML components under real-time, power, and safety constraints. It reviews emerging methodologies, architectures, and frameworks spanning design- and run-time assurance, with emphasis on advances in reliability modeling, secure system design, and certification approaches that aim to accommodate imperfect, learning-enabled components and thereby bridge the gap to certifiable system-level dependability.
Significance. The topic is timely and relevant to the cs.AI community given the rapid deployment of AI in safety-critical domains. A balanced survey that synthesizes challenges across abstraction layers and points to concrete research directions could help researchers and practitioners navigate the gap between AI capabilities and certification requirements. The manuscript's value lies in its holistic framing rather than in new derivations, empirical results, or machine-checked proofs; its impact will therefore depend on the depth and critical balance of the reviewed approaches.
minor comments (4)
- [Abstract / Introduction] The abstract and introduction repeat the phrase 'dependability' and 'certifiable system-level dependability' multiple times without defining the precise scope of 'dependability' (safety + security + reliability + certification) or distinguishing between component-level and system-level guarantees; a short clarifying paragraph or footnote would improve precision.
- The discussion of emerging frameworks would be strengthened by a comparative table (or diagram) that maps specific methodologies to the four pillars (safety, security, reliability, certification) and notes their handling of non-determinism, data dependence, and real-time constraints; without such a summary the review remains somewhat diffuse.
- [Introduction] Several claims about the limitations of traditional methods (e.g., failure to address 'dynamic and uncertain behaviors') would benefit from one or two concrete, referenced examples from the autonomous-systems literature rather than remaining at a general level.
- The manuscript should explicitly state its review methodology (e.g., search terms, time window, inclusion criteria) and acknowledge any scope limitations (e.g., focus on automotive vs. broader robotics or avionics) to allow readers to assess completeness.
Simulated Author's Rebuttal
We thank the referee for their positive and accurate summary of our manuscript, as well as for recommending minor revision. We appreciate the acknowledgment that the topic is timely for the cs.AI community and that a holistic survey synthesizing challenges across safety, security, reliability, and certification can help bridge AI capabilities with certification needs. The referee's description correctly captures our focus on the shortcomings of traditional methods when applied to non-deterministic AI/ML components and our review of emerging design- and run-time assurance approaches.
Circularity Check
No significant circularity identified
full rationale
The manuscript is a survey paper that reviews design challenges for dependability in AI-integrated autonomous systems. It contains no equations, derivations, fitted parameters, predictions, or quantitative models. The central claims are descriptive statements about limitations of traditional methods and the existence of emerging approaches; these are not derived from or reduced to any self-referential inputs, self-citations, or ansatzes within the paper. No load-bearing step reduces by construction to the paper's own content, satisfying the criteria for a self-contained discussion without circularity.
Axiom & Free-Parameter Ledger
Reference graph
Works this paper leans on
-
[1]
A systematic review on risk management and enhancing reliability in autonomous vehicles,
A. Mahmood and R. Szabolcsi, “A systematic review on risk management and enhancing reliability in autonomous vehicles,”Machines, vol. 13, no. 8, 2025
work page 2025
-
[2]
Computing systems for autonomous driving: State of the art and challenges,
L. Liuet al., “Computing systems for autonomous driving: State of the art and challenges,”IEEE Internet of Things Journal, vol. 8, no. 8, pp. 6469–6486, 2021
work page 2021
-
[3]
B. Ranjbaret al., “Learning-oriented reliability improvement of com- puting systems from transistor to application level,” inProc. of Design, Automation & Test in Europe Conference & Exhibition (DATE), 2023
work page 2023
-
[4]
Reliability-aware resource management in multi-/many-core systems: A perspective paper,
S. S. Sahoo, B. Ranjbar, and A. Kumar, “Reliability-aware resource management in multi-/many-core systems: A perspective paper,”Journal of Low Power Electronics and Applications, vol. 11, no. 1, p. 7, 2021
work page 2021
-
[5]
Early reliability assessment of ai-based automotive systems,
S. Hegdeet al., “Early reliability assessment of ai-based automotive systems,”ACM Trans. Internet Things, 2025
work page 2025
-
[6]
Reliability and safety assessment of automated driving systems: Review and preview,
K.-W. Wu, C.-C. Liao, and W.-F. Wu, “Reliability and safety assessment of automated driving systems: Review and preview,” inProc. of IEEE International Conference on Industrial Engineering and Engineering Management (IEEM), 2020, pp. 390–394
work page 2020
-
[7]
Mixed criticality systems—a history of misconceptions?
R. Ernst and M. Di Natale, “Mixed criticality systems—a history of misconceptions?”IEEE Design & Test, vol. 33, no. 5, pp. 65–74, 2016
work page 2016
-
[8]
ISO 26262- road vehicles — functional safety,
“ISO 26262- road vehicles — functional safety,” International Organiza- tion for Standardization (ISO), Standard, Dec. 2018
work page 2018
-
[9]
Dynamic reliability management for multi-core processor based on deep reinforcement learning,
Z. Sun, H. Zhou, and S. X.-D. Tan, “Dynamic reliability management for multi-core processor based on deep reinforcement learning,” inProc. of Intl. Conference on Synthesis, Modeling, Analysis and Simulation Methods and Applications to Circuit Design (SMACD), 2019
work page 2019
-
[10]
Mapping on multi/many-core systems: Survey of current and emerging trends,
A. K. Singh, M. Shafique, A. Kumar, and J. Henkel, “Mapping on multi/many-core systems: Survey of current and emerging trends,” in Proc. of ACM/IEEE Design Automation Conference (DAC), 2013
work page 2013
-
[11]
Peak-power aware life-time reliability improvement in fault-tolerant mixed-criticality systems,
M. Navardiet al., “Peak-power aware life-time reliability improvement in fault-tolerant mixed-criticality systems,”IEEE Open Journal of Circuits and Systems, vol. 3, pp. 199–215, 2022
work page 2022
-
[12]
FANTOM: Fault tolerant task-drop aware scheduling for mixed-criticality systems,
B. Ranjbar, B. Safaei, A. Ejlali, and A. Kumar, “FANTOM: Fault tolerant task-drop aware scheduling for mixed-criticality systems,”IEEE ACCESS, vol. 8, pp. 187 232–187 248, 2020
work page 2020
-
[13]
B. Ranjbar, A. Hosseinghorban, M. Salehi, A. Ejlali, and A. Kumar, “Toward the design of fault-tolerance-aware and peak-power-aware mul- ticore mixed-criticality systems,”IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems (TCAD), vol. 41, no. 5, pp. 1509–1522, 2022
work page 2022
-
[14]
S. S. Sahoo, B. Veeravalli, and A. Kumar, “A hybrid agent-based design methodology for dynamic cross-layer reliability in heterogeneous em- bedded systems,” inProc. of ACM/IEEE Design Automation Conference (DAC), 2019
work page 2019
-
[15]
Cross-layer fault-tolerant design of real-time systems,
——, “Cross-layer fault-tolerant design of real-time systems,” inProc. of IEEE International Symposium on Defect and Fault Tolerance in VLSI and Nanotechnology Systems (DFT), 2016, pp. 63–68
work page 2016
-
[16]
——, “CL(R)early: An early-stage DSE methodology for cross- layer reliability-aware heterogeneous embedded systems,” inProc. of ACM/IEEE Design Automation Conference (DAC), 2020
work page 2020
-
[17]
Power management of multicore systems,
B. Ranjbar, A. K. Singh, S. S. Sahoo, P. Dziurzanski, and A. Kumar, “Power management of multicore systems,” inHandbook of Computer Architecture. Springer, 2023, pp. 1–33
work page 2023
-
[18]
S. S. Sahoo, A. Das, and A. Kumar, “Fault tolerant architectures,” in Handbook of Computer Architecture. Springer, 2023, pp. 1–44
work page 2023
-
[19]
S. S. Sahooet al., “Emergent design challenges for embedded systems and paths forward: mixed-criticality, energy, reliability and security perspectives,” inProc. of International Conference on Hardware/Software Codesign and System Synthesis (CODES+ISSS), 2021
work page 2021
-
[20]
Roadmap for cyberse- curity in autonomous vehicles,
V . K. Kukkala, S. V . Thiruloga, and S. Pasricha, “Roadmap for cyberse- curity in autonomous vehicles,”IEEE Consumer Electronics Magazine, vol. 11, no. 6, pp. 13–23, 2022
work page 2022
-
[21]
Remote exploitation of an unaltered passenger vehicle,
C. Miller and C. Valasek, “Remote exploitation of an unaltered passenger vehicle,”Black Hat USA, vol. 2015, no. S 91, pp. 1–91, 2015
work page 2015
-
[22]
Sedan: Security-aware design of time-critical automotive networks,
V . K. Kukkala, S. Pasricha, and T. Bradley, “Sedan: Security-aware design of time-critical automotive networks,”IEEE Transactions on Vehicular Technology, vol. 69, no. 8, pp. 9017–9030, 2020
work page 2020
-
[23]
Indra: Intrusion detection using recurrent autoencoders in automotive embedded systems,
V . K. Kukkala, S. V . Thiruloga, and S. Pasricha, “Indra: Intrusion detection using recurrent autoencoders in automotive embedded systems,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 39, no. 11, pp. 3698–3710, 2020
work page 2020
-
[24]
Latte: LSTM self-attention based anomaly detection in embed- ded automotive platforms,
——, “Latte: LSTM self-attention based anomaly detection in embed- ded automotive platforms,”ACM Transactions on Embedded Computing Systems (TECS), vol. 20, no. 5s, pp. 1–23, 2021
work page 2021
-
[25]
TENET: Temporal CNN with attention for anomaly detection in automotive cyber-physical systems,
S. V . Thiruloga, V . K. Kukkala, and S. Pasricha, “TENET: Temporal CNN with attention for anomaly detection in automotive cyber-physical systems,” inProc. of Asia and South Pacific Design Automation Confer- ence (ASP-DAC), 2022
work page 2022
-
[26]
Anomaly detection in autonomous vehicle’s lidar sensor data using variational autoencoders,
N. Sboui, M. Hadded, H. Ghazzai, M. Elhadef, and G. Setti, “Anomaly detection in autonomous vehicle’s lidar sensor data using variational autoencoders,” inProc. of IEEE 100th Vehicular Technology Conference (VTC2024-Fall), 2024, pp. 1–5
work page 2024
-
[27]
S. Rezaei, N. Masoud, and A. Khojandi, “GAAD: GAN-enabled autoen- coder for real-time sensor anomaly detection and recovery in autonomous driving,”IEEE Sensors Journal, vol. 24, no. 7, pp. 11 734–11 742, 2024
work page 2024
-
[28]
The ghost navigator: Revisiting the hidden vulnerability of localization in autonomous driving,
J. Zhanget al., “The ghost navigator: Revisiting the hidden vulnerability of localization in autonomous driving,” inProc. of USENIX Security Symposium (USENIX Security 25), 2025, pp. 3979–3998
work page 2025
-
[29]
GPS-IDS: An anomaly-based GPS spoofing attack detection framework for autonomous vehicles,
M. M. Abraret al., “GPS-IDS: An anomaly-based GPS spoofing attack detection framework for autonomous vehicles,”arXiv preprint arXiv:2405.08359, 2024
-
[30]
P. A. Bonab and A. Sargolzaei, “A secure cooperative adaptive cruise control design with unknown leader dynamics under false data injection attacks,”Computers, vol. 14, no. 3, p. 84, 2025
work page 2025
-
[31]
Attacks and defences on intelligent connected vehicles: A survey,
M. Dibaeiet al., “Attacks and defences on intelligent connected vehicles: A survey,”Digital Communications and Networks, vol. 6, no. 4, pp. 399– 421, 2020
work page 2020
-
[32]
Challenges in automotive cyber-physical systems design,
D. Goswamiet al., “Challenges in automotive cyber-physical systems design,” inInternational Conference on Embedded Computer Systems: Architectures, Modeling and Simulation (ICSAMOS), 2012
work page 2012
-
[33]
Autonomy-driven emerging directions in software- defined vehicles,
U. D. Bordoloiet al., “Autonomy-driven emerging directions in software- defined vehicles,” inProc. of Design, Automation & Test in Europe Conference & Exhibition (DATE), 2023
work page 2023
-
[34]
Timing predictability for SOME/IP-based service- oriented automotive in-vehicle networks,
E. Fraccaroliet al., “Timing predictability for SOME/IP-based service- oriented automotive in-vehicle networks,” inProc. of Design, Automation & Test in Europe Conference & Exhibition (DATE), 2023
work page 2023
-
[35]
Statistical verification of autonomous system controllers under timing uncertainties,
B. Ghoshet al., “Statistical verification of autonomous system controllers under timing uncertainties,”Real-Time Systems, vol. 60, no. 1, pp. 108– 149, 2024
work page 2024
-
[36]
J.-H. Oetjenset al., “Safety evaluation of automotive electronics using virtual prototypes: State of the art and research challenges,” inProc. of ACM/IEEE Design Automation Conference (DAC), 2014
work page 2014
-
[37]
Co-design of cyber- physical systems via controllers with flexible delay constraints,
D. Goswami, R. Schneider, and S. Chakraborty, “Co-design of cyber- physical systems via controllers with flexible delay constraints,” inProc. of Asia and South Pacific Des. Aut. Conf. (ASP-DAC), 2011
work page 2011
-
[38]
Schedule integration framework for time-triggered automotive architectures,
F. Sagstetteret al., “Schedule integration framework for time-triggered automotive architectures,” inProc. of Design Aut. Conf. (DAC), 2014
work page 2014
-
[39]
Testing automotive embedded systems under x-in-the- loop setups,
G. Tibbaet al., “Testing automotive embedded systems under x-in-the- loop setups,” inIEEE/ACM International Conference on Computer-Aided Design (ICCAD), 2016
work page 2016
-
[40]
Learning-enabled CPS for edge-cloud computing,
L. Capogrossoet al., “Learning-enabled CPS for edge-cloud computing,” inInternational Symposium on Industrial Embedded Systems (SIES), 2024, pp. 132–139
work page 2024
-
[41]
LO-SC: Local-only split computing for accurate deep learning on edge devices,
——, “LO-SC: Local-only split computing for accurate deep learning on edge devices,” inProc. of International Conference on VLSI Design (VLSID), 2025, pp. 445–450
work page 2025
-
[42]
Safety Analysis of Embedded Controllers Under Implementation Platform Timing Uncertainties,
C. Hobbset al., “Safety Analysis of Embedded Controllers Under Implementation Platform Timing Uncertainties,”IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems (TCAD), vol. 41, no. 11, pp. 4016–4027, 2022
work page 2022
-
[43]
Constraint-driven synthesis and tool-support for FlexRay-based automotive control systems,
R. Schneideret al., “Constraint-driven synthesis and tool-support for FlexRay-based automotive control systems,” inProc. of Intl. Conference on Hw/Sw Codesign and System Synthesis (CODES+ISSS), 2011
work page 2011
-
[44]
Controllers for edge-cloud cyber-physical systems,
T. Zhuet al., “Controllers for edge-cloud cyber-physical systems,” in Proc. of Int. Conf. on Comm. Systems and Networks (COMSNETS), 2025
work page 2025
-
[45]
Designing imperfect cyber-physical systems,
S. Chakraborty and K. Schneider, “Designing imperfect cyber-physical systems,” inForum on Specification and Design Languages (FDL), 2025
work page 2025
-
[46]
P. Ganguliet al., “Trading delays with uncertainty: Controller design for DNN-based perception processing on edge-cloud platforms,” inProc. of 33rd Intl. Conference on Real-Time Networks and Systems (RTNS), 2025
work page 2025
-
[47]
Safety-driven DNN sizing for vehicular CPS,
T. Zhuet al., “Safety-driven DNN sizing for vehicular CPS,”IEEE Embedded Systems Letters, 2025 (early access), https://doi.org/10.1109/LES.2025.3595839
-
[48]
Quantitative safety-driven co-synthesis of cyber-physical system implementations,
C. Hobbset al., “Quantitative safety-driven co-synthesis of cyber-physical system implementations,” inProc. of International Conference on Cyber- Physical Systems (ICCPS), 2024
work page 2024
-
[49]
Poster abstract: Neural architecture sizing for autonomous systems,
S. Xuet al., “Poster abstract: Neural architecture sizing for autonomous systems,” inProc. of International Conference on Cyber-Physical Systems (ICCPS), 2024
work page 2024
-
[50]
Enhancing split computing and early exit applica- tions through predefined sparsity,
L. Capogrossoet al., “Enhancing split computing and early exit applica- tions through predefined sparsity,” inForum on Specification and Design Languages (FDL), 2024
work page 2024
-
[51]
GPU partitioning & neural architecture sizing for safety- driven sensing in autonomous systems,
S. Xuet al., “GPU partitioning & neural architecture sizing for safety- driven sensing in autonomous systems,” inProc. of International Con- ference on Autonomous Agents (ICAA), 2024, pp. 67–76
work page 2024
-
[52]
Rate analysis for streaming applications with on- chip buffer constraints,
A. Maxiaguineet al., “Rate analysis for streaming applications with on- chip buffer constraints,” inProc. of Asia and South Pacific Design Autom. Conf. (ASP-DAC), 2004, pp. 131–136
work page 2004
-
[53]
Re-engineering cyber- physical control applications for hybrid communication protocols,
D. Goswami, R. Schneider, and S. Chakraborty, “Re-engineering cyber- physical control applications for hybrid communication protocols,” in Proc. of Design, Automation & Test in Europe Conference & Exhibition (DATE), 2011
work page 2011
-
[54]
Safety-aware flexible schedule synthesis for cyber-physical systems using weakly-hard constraints,
S. Xuet al., “Safety-aware flexible schedule synthesis for cyber-physical systems using weakly-hard constraints,” inProc. of Asia and South Pacific Design Automation Conference (ASP-DAC), 2023
work page 2023
-
[55]
Repairing control safety violations via scheduler patch synthesis,
A. Yeolekaret al., “Repairing control safety violations via scheduler patch synthesis,” inProc. of International Conference on Cyber-Physical Systems (ICCPS), 2025
work page 2025
-
[56]
MTL-Split: Multi-Task Learning for Edge Devices using Split Computing,
L. Capogrossoet al., “MTL-Split: Multi-Task Learning for Edge Devices using Split Computing,” inProc. of Design Aut. Conf. (DAC), 2024
work page 2024
-
[57]
Lightweight authentication for secure automotive networks,
P. Mundhenket al., “Lightweight authentication for secure automotive networks,” inProc. of Des, Aut. & Test in Europe Conf. (DATE), 2015
work page 2015
-
[58]
P. Waszeckiet al., “Automotive electrical and electronic architecture security via distributed in-vehicle traffic monitoring,”IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 36, no. 11, pp. 1790–1803, 2017
work page 2017
-
[59]
“Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act),” 2024. [Online]. Available: https://eur-lex.europa.eu/eli/reg/2024/1689/oj
work page 2024
-
[60]
Updated architecture of standards,
“Updated architecture of standards,” The European Committee for Standardization (CEN) and the European Committee for Electrotechnical Standardization (CENELEC), Standard, 2024. [Online]. Available: https://jtc21.eu/
work page 2024
-
[61]
Quantifying uncertainty in environmental sensing with evidential deep learning,
S. Mittermaier, S. Patra, and C. Carbonelli, “Quantifying uncertainty in environmental sensing with evidential deep learning,” inIEEE SENSORS, 2023, pp. 1–4
work page 2023
-
[62]
S. Seifi, T. Sukianto, C. Carbonelli, L. Servadei, and R. Wille, “Comply- ing with the eu ai act: Innovations in explainable and user-centric hand gesture recognition,”Machine Learning with Applications, 2025
work page 2025
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.