Pith. sign in

REVIEW 3 major objections 6 minor 114 references

The DevSafeOps Dilemma: A Systematic Literature Review on Rapidity in Safe Autonomous Driving Development and Operation

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A systematic literature review claims that applying DevOps to safety-related autonomous driving is not a deadlock but a mapped problem space, synthesising 319 studies into 11 challenge clusters and matching each to candidate solutions…

desk verdict A competent SLR that usefully maps DevOps-for-safe-AD challenges, but the map is less solid than the conclusions suggest—worth reviewing with required revisions. read the letter →

arxiv 2506.21693 v1 pith:CS2IPSWL submitted 2025-06-26 cs.SE cs.RO

Ali Nouri , Beatriz Cabrero-Daniel , Fredrik Törner , Christian Berger This is my paper · ORCID
classification cs.SEcs.RO
keywords DevOpsautonomousdrivingMLOpsSOTIFISO26262Safesafetyargumentationsystematicliteraturereview
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that the tension between DevOps speed and automotive safety is not a deadlock but a structured problem space. It introduces DevSafeOps, a term for integrating safety-related activities into the DevOps iterative loop, and uses a systematic literature review to synthesise existing work. The review identifies 11 clusters of challenges, from workflow separation to cross-domain impact, and maps proposed solutions to those challenges using only links the original papers make. It concludes that several open topics remain, meaning the map is a starting point rather than a finished solution. A sympathetic reader would take away that practitioners now have a catalogue of where DevOps and safety collide and what has been tried.

What carries the argument

The DevSafeOps loop is the central object: a reinterpretation of the DevOps infinity loop in which safety activities such as hazard analysis, verification, validation, and safety argumentation are performed in every iteration. The review's 11 challenge clusters are aligned with phases of ISO 26262 and ISO 21448, and each cluster is paired with the solutions that the literature proposes, with a table indicating which paper made each mapping. The mapping table also marks which solutions have not been mapped to a specific challenge, revealing gaps that answer the third research question about open topics.

What would settle it

Run the same systematic search with additional terms such as 'continuous safety assurance', 'agile safety', 'safety case automation', or 'continuous certification', and count whether the newly retrieved primary studies introduce challenges outside the eleven clusters; a finding of one or more genuinely new clusters would show the map is incomplete.

Watch

Extended reading notes

Core claim

The paper claims that the safety community and the DevOps community can be reconciled, provided safety activities are embedded into every stage of the continuous development and operation loop. Based on a systematic review of 319 extracted studies, it reports 11 clusters of challenges grouped according to the safety activities prescribed by ISO 26262, ranging from requirements updates and safety analysis to safety argumentation, certification, and tooling. For each challenge it lists candidate solutions that appear in the reviewed literature, such as SafeScrum-style embedded quality roles, simulation and shadow-mode testing, continuous safety case fragments, contract-based design, and LLM-assisted analysis. It also explicitly maps which challenges remain open, including requirements engineering for DevOps, hardware aspects of continuous updates, and cross-cutting solutions that were suggested for one challenge but not yet validated for others.

Load-bearing premise

The map is only as complete as the literature search: the query insists on the words 'safety', one of the standard identifiers 26262, 21448 or SOTIF, and 'DevOps' or 'MLOps', so studies that use different vocabulary, such as 'continuous safety assurance' or 'agile safety', could be missing and could change the clusters.

Editorial extensions

If this is right

  • Practitioners gain a structured catalogue of 11 challenge clusters, from workflow separation to cross-domain impact, each tied to safety lifecycle phases and candidate mitigation strategies.
  • Solution mappings are only accepted where the original literature explicitly links them, so the gaps in the tables are direct evidence of where evidence is missing.
  • Open topics named by the review include requirements engineering for DevOps, hardware aspects of continuous updates, and cross-cutting solutions such as contract-based design and LLM-based automation.
  • If the identified solutions are adopted, the review argues that development speed and safety can be combined, with speed bounded by safety requirements rather than sacrificed.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The eleven clusters likely generalise beyond autonomous driving to other safety-critical domains adopting DevOps, since most challenges are phrased in standard safety-process terms rather than vehicle-specific terms.
  • The review's own gap analysis implies that contract-based design and LLM-based automation, currently listed as solutions to specific challenges, could be evaluated as cross-cutting enablers; the paper notes this possibility but leaves it unstudied.
  • A quantitative follow-up could measure, per cluster, how many industrial Voluntary Safety Self-Assessment reports mention the challenge, turning the qualitative map into a priority list for standardisation bodies.
  • If the open topics the paper identifies are resolved, the next bottleneck is likely to shift from individual safety activities to toolchain integration across those activities.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. This paper reports a systematic literature review, following Kitchenham and Charters, on challenges and solutions in applying DevOps/MLOps to safety-related autonomous driving functions. The authors introduce the term DevSafeOps, synthesize 11 challenge clusters (CH1-CH11) mapped onto an adapted DevOps loop, map proposed solutions to challenges, and use VSSA reports for industrial context. The central claim is that the literature shows a structured landscape of challenges and solution candidates, with several open topics for future research.

Significance. If the map is accepted, it provides a useful catalog for researchers and practitioners: named challenge clusters, mapped solutions with source attribution, inter-challenge dependencies, and industrial practice. The paper has real strengths: a systematic protocol, dual screening, snowballing, a supplementary data set, explicit marking of industrial and non-peer-reviewed sources, and a design that avoids inventing solution mappings beyond what the primary studies propose. The contribution is mainly a synthesis and map rather than a new technical solution, so its value depends on the robustness and representativeness of the included evidence; the major comments below focus on that dependency.

major comments (3)
  1. [Tables 1-3 and Section 5] The support for the 11-cluster map is thin and uneven, and the conclusion in Section 6 that the '319 extracted studies' show 11 challenge clusters overstates the evidence actually cited in the mapping tables. CH11, for example, has a single challenge reference ([73], co-authored by the review team) and a single solution reference ([2]); CH8 has two challenge references and one solution reference; and CH10 draws several of its challenge statements and both proposed solutions from [88], a non-peer-reviewed arXiv position paper. Because the review deliberately maps only those solutions that original sources proposed (Section 4.3), the resulting map is closer to a re-statement of a small set of proposals than to an independent synthesis. Please add a per-challenge evidence table reporting the number of distinct study groups, publication types, and author-affiliated references, and calibrate the conclusions (for example, 'the literature supports' should become 'the included studies suggest') wherever support rests on one or two sources.
  2. [Section 4.1] The search query requires the literal co-occurrence of 'safety' with '26262 OR 21448 OR SOTIF' and 'DevOps OR MLOps' together with automotive keywords, so work framed as 'continuous safety assurance', 'continuous assurance cases', 'SafeOps', 'agile safety', or 'safety case automation' is likely under-represented even when it addresses the same research questions. The paper's reliance on snowballing (Figure 1) and on a VSSA industrial supplement (Section 4.4) suggests that the database query alone has limited recall. This matters because RQ1 and RQ2 are completeness claims about the literature; if a supplementary search with the alternative terminology cannot be run, the paper should explicitly bound its conclusions to work using DevOps/MLOps terminology and report which included primary studies came from the database query versus snowballing.
  3. [Section 6 (RQ3)] The answer to RQ3 is not derived transparently from the primary studies. The conclusion identifies open topics through author-level synthesis ('we have identified gaps in each challenge', 'This highlights the need for further research') and through cross-challenge extrapolation, but no table, code, or extraction rule indicates which primary studies state which open challenges. Since RQ3 is one of the paper's three research questions, please define the criterion for an 'open challenge', report the source for each open topic, and explicitly separate literature-reported gaps from the authors' own inferences.
minor comments (6)
  1. [Section 4.2] The word 'defiend' should be 'defined'.
  2. [Section 2.1] The text refers to 'ISO 4804' in one place and 'ISO/TR 4804' elsewhere; please make the designation consistent.
  3. [Section 6] The identifiers 'Sol3.3' and 'Test automation (Sol5.3)' refer to solutions that do not exist as numbered; the first is likely 'Sol9.1' or 'Sol3.2', and the second should be 'Sol5.4'.
  4. [Tables 1-3] The header 'CHX & CH. Ref.' and the column labeled 'Justification of mapping' are confusing; the latter actually contains the solution proposal itself, not a justification of the mapping. Rename or split the columns.
  5. [Section 2 (CH7)] The name 'Munk and Schweiser' should be 'Munk and Schweizer' to match reference [70].
  6. [Section 4.1] The text says 'four literature databases' but then names three databases and Google Scholar; please reword or name the fourth.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the review is an interpretive synthesis, not a fitted prediction or a renamed input.

full rationale

The paper is a systematic literature review that identifies challenges and solutions from primary studies and groups them into 11 thematic clusters. The clustering is explicitly presented as an organizational step applied after data extraction (“clustering was done for presentation and communication purposes, and it does not influence the data extraction process”, Section 4.5), so the cluster structure is not used to derive the data it claims to synthesize. No equations or fitted parameters are present, and no quantity is predicted from a fitted value. The authors introduce the term DevSafeOps, but this is a naming choice, not a derivation; the review’s conclusions do not depend on the term being true. Several self-citations appear as primary studies (e.g., [73], [74], [75], [76]), and CH11 is supported solely by [73], a prior peer-reviewed industrial experience report co-authored by the review team. This is a legitimate evidential-thinness concern, but it is not circularity: [73] is a distinct prior study, not the present paper’s own output, and the present paper does not define or justify its conclusions in terms of that citation. The paper also discloses its inclusion of non-peer-reviewed sources ([88], [104]) and the need for future validation. Therefore, the derivation chain is self-contained as a literature synthesis, and no circular step can be exhibited.

Assumptions & free parameters 0 free parameters · 3 assumptions · 1 invented entities

This is a literature review, so the ledger contains no fitted parameters. The main assumptions are about the completeness of the search, the validity of structuring challenges around ISO 26262 abstraction levels, and the credibility of VSSA reports as evidence of industrial practice. One invented entity, the DevSafeOps label, is a terminology contribution with no independent falsifiable handle.

assumptions (3)
  • domain assumption The search query and selected databases (ACM, IEEE, Scopus, Google Scholar) retrieve a representative and sufficiently complete set of primary studies on safety-related DevOps in automotive.
    The validity of the SLR's conclusions depends on this; authors mitigate with snowballing and grey literature, but completeness is acknowledged as a threat in Section 4.5.
  • domain assumption ISO 26262 abstraction levels provide an appropriate and non-biasing structure for clustering the identified challenges.
    The authors state in Section 4.5 that clustering was done for presentation and does not influence conclusions; however, the choice determines which activities are highlighted.
  • domain assumption Voluntary Safety Self-Assessment (VSSA) reports and other grey literature are credible sources of industrial practice and safety posture.
    Section 4.4 introduces VSSA reports as complementary evidence; their accuracy is not independently verified.
invented entities (1)
  • DevSafeOps
    purpose: Label for the integration of safety activities into DevOps loops; used as the organizing concept for the review.
    It organizes the synthesis but is not a testable physical or mathematical entity. It is a terminology contribution rather than a model with falsifiable predictions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of The DevSafeOps Dilemma: A Systematic Literature Review on Rapidity in Safe Autonomous Driving Development and Operation." pith.science (2026). https://pith.science/paper/CS2IPSWL

@misc{pith2026250621693,
  author       = {Pith},
  title        = {Pith review of: The DevSafeOps Dilemma: A Systematic Literature Review on Rapidity in Safe Autonomous Driving Development and Operation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CS2IPSWL}},
  note         = {Machine review of arXiv:2506.21693}
}
read the original abstract

Developing autonomous driving (AD) systems is challenging due to the complexity of the systems and the need to assure their safe and reliable operation. The widely adopted approach of DevOps seems promising to support the continuous technological progress in AI and the demand for fast reaction to incidents, which necessitate continuous development, deployment, and monitoring. We present a systematic literature review meant to identify, analyse, and synthesise a broad range of existing literature related to usage of DevOps in autonomous driving development. Our results provide a structured overview of challenges and solutions, arising from applying DevOps to safety-related AI-enabled functions. Our results indicate that there are still several open topics to be addressed to enable safe DevOps for the development of safe AD.

Figures

Figures reproduced from arXiv: 2506.21693 by the authors.

Figure 1
Figure 1. Flow diagram depicting the application of inclusion and exclusion criteria. [PITH_FULL_IMAGE:figures/full_fig_p009_1.png] view at source ↗
Figure 2
Figure 2. The figure outlines the safety activities that are necessary to be carried out in DevSafeOps cycles, along with the [PITH_FULL_IMAGE:figures/full_fig_p014_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

114 extracted references · 68 canonical work pages

  1. [73]

    Nouri, A., Berger, C., T¨ orner, F., 2022. An Industrial Experience Re- port about Challenges from Continuous Monitoring, Improvement, and Deployment for Autonomous Driving Features, in: Euromicro Confer- ence on Software Engineering and Advanced Applications, pp. 358–365. doi:10.1109/SEAA56994.2022.00063

    arXiv 2022

  2. [2]

    Agirre, I., Onaindia, P., Poggi, T., Yarza, I., Cazorla, F.J., Kosmidis, L., Gr¨ uttner, K., Abuteir, M., Loewe, J., Orbegozo, J.M., Botta, S.,

  3. [88]

    SafetyOps

    Siddique, U., 2020. SafetyOps. arXiv preprint arXiv:2008.04461

    work page Pith review arXiv 2020

  4. [1]

    Safety Report a2z Smart City Solution

    a2z, . Safety Report a2z Smart City Solution. URL: https://autoa2z. ai/Safety. accessed: 2025-02-28

    2025

  5. [3]

    Amalfitano, D., De Luca, M., Fasolino, A.R., Pelliccione, P., Santilli, T., 2024. Characterizing Software Architectural Metrics for Continu- ous Compliance in the Automotive Domain, in: 2024 IEEE 21st In- ternational Conference on Software Architecture (ICSA), IEEE. pp. 182–193

    2024

  6. [4]

    Our Approach to Automated Driving System Safety

    Apple, . Our Approach to Automated Driving System Safety. URL: https://www.apple.com/ads/ADS-Safety.pdf. accessed: 2025-02- 28

    2025

  7. [5]

    Aurora Safety Report

    Aurora, a. Aurora Safety Report. URL: https://aurora.tech/ safety. accessed: 2025-02-28

    2025

  8. [6]

    Aurora’s Safety Case Framework

    Aurora, b. Aurora’s Safety Case Framework. URL: https:// safetycaseframework.aurora.tech/gsn. accessed: 2025-02-28

    2025

Show all 114 references
  1. [7]

    Data-Driven Development Process, The AutoX Team

    AutoX, . Data-Driven Development Process, The AutoX Team. URL: https://www.autox.ai/blog/20200226.html. accessed: 2025-02-28

    2025

  2. [8]

    A VSC Best Practice for Core Automated Ve- hicle Safety Information

    A VSC-D-02-2024, 2024. A VSC Best Practice for Core Automated Ve- hicle Safety Information. Standard. SAE. 38

    2024

  3. [9]

    Architectural Considerations in the Certifi- cation of Modular Systems, in: Anderson, S., Felici, M., Bologna, S

    Bate, I., Kelly, T., 2002. Architectural Considerations in the Certifi- cation of Modular Systems, in: Anderson, S., Felici, M., Bologna, S. (Eds.), Computer Safety, Reliability and Security, pp. 321–333

    2002

  4. [10]

    (not) yet another meta- model for traceability, in: 2021 ACM/IEEE International Confer- ence on Model Driven Engineering Languages and Systems Compan- ion (MODELS-C), pp

    Batot, E.R., Cabot, J., G´ erard, S., 2021. (not) yet another meta- model for traceability, in: 2021 ACM/IEEE International Confer- ence on Model Driven Engineering Languages and Systems Compan- ion (MODELS-C), pp. 787–796. doi: 10.1109/MODELS-C53483.2021. 00125

    2021

  5. [11]

    Predicting the type of road traffic accident for test scenario generation

    B¨ aumler, M., Prokop, G., 2024. Predicting the type of road traffic accident for test scenario generation. IEEE Access

    2024

  6. [12]

    Contracts for system design

    Benveniste, A., Caillaud, B., Nickovic, D., Passerone, R., Raclet, J.B., Reinkemeier, P., Sangiovanni-Vincentelli, A., Damm, W., Henzinger, T.A., Larsen, K.G., et al., 2018. Contracts for system design. Founda- tions and Trends® in Electronic Design Automation 12, 124–400

    2018

  7. [13]

    A compre- hensive survey on Software as a Service (SaaS) transformation for the automotive systems

    Blanco, D.F., Le Mou¨ el, F., Lin, T., Escudi´ e, M.P., 2023. A compre- hensive survey on Software as a Service (SaaS) transformation for the automotive systems. IEEE Access

    2023

  8. [14]

    Borg, M., 2021. The AIQ meta-testbed: Pragmatically bridging aca- demic AI testing and industrial Q needs, in: Software Quality: Future Perspectives on Software Engineering Quality: 13th International Con- ference, SWQD 2021, Vienna, Austria, January 19–21, 2021, Proceed- ings...

    2021

  9. [15]

    Safely Entering the Deep: A Review of Verification and Validation for Machine Learning and a Challenge Elicitation in the Automotive Industry

    Borg, M., Englund, C., Wnuk, K., Duran, B., Levandowski, C., Gao, S., Tan, Y., Kaijser, H., L¨ onn, H., T¨ ornqvist, J., 2019. Safely Entering the Deep: A Review of Verification and Validation for Machine Learning and a Challenge Elicitation in the Automotive Industry. Journal...

    2019 doi

  10. [16]

    Borg, M., Jabangwe, R., ˚Aberg, S., Ekblom, A., Hedlund, L., Lidfeldt, A., 2021. Test automation with grad-cam heatmaps - a future pipe segment in mlops for vision ai?, in: 2021 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), p...

    2021

  11. [17]

    Accelerating Digital Transformation: 10 Years of Software Center

    Bosch, J., Carlson, J., Olsson, H.H., Sandahl, K., Staron, M., 2022. Accelerating Digital Transformation: 10 Years of Software Center. Springer Nature

    2022

  12. [18]

    It Takes Three to Tango: Requirement, Outcome/Data, and AI Driven Development, in: Hyryn- salmi, S., Suominen, A., Jud, C., Wang, X., Bosch, J., M¨ unch, J

    Bosch, J., Olsson, H.H., Crnkovic, I., 2018. It Takes Three to Tango: Requirement, Outcome/Data, and AI Driven Development, in: Hyryn- salmi, S., Suominen, A., Jud, C., Wang, X., Bosch, J., M¨ unch, J. (Eds.), Proceedings of the International Workshop on Software- intensive Bu...

    2018

  13. [19]

    Model Reporting for Certifiable AI: A Proposal from Merging EU Regulation into AI Development

    Brajovic, D., Renner, N., Goebels, V.P., Wagner, P., Fr´ esz, B., Biller, M., Klaeb, M., Kutz, J., Neuhuettler, J., Huber, M.F., 2023. Model Reporting for Certifiable AI: A Proposal from Merging EU Regulation into AI Development. ArXiv abs/2307.11525. URL: https://api. semanti...

    2023 arXiv

  14. [20]

    C´ espedes, D., Angeleri, P., Melendez, K., D´ avila, A., 2020. Software product quality in devops contexts: A systematic literature review, in: Trends and Applications in Software Engineering: Proceedings of the 8th International Conference on Software Process Improvement (CI...

    2020

  15. [21]

    System Theoretic Process Analysis (STPA) Recommended Practices for Evaluations of Automotive Related Safety- Critical Systems

    Committee, F.S., 2022. System Theoretic Process Analysis (STPA) Recommended Practices for Evaluations of Automotive Related Safety- Critical Systems. https://doi.org/10.4271/J3187_202202

    2022 doi

  16. [22]

    Requirements Engineering in the Age of Societal- Scale Cyber-Physical Systems: The Case of Automated Driving, in: IEEE International Requirements Engineering Conference, IEEE

    Czarnecki, K., 2018. Requirements Engineering in the Age of Societal- Scale Cyber-Physical Systems: The Case of Automated Driving, in: IEEE International Requirements Engineering Conference, IEEE. pp. 3–4. URL: https://ieeexplore.ieee.org/abstract/document/ 8491118/

    2018

  17. [23]

    Czarnecki, K., 2019. Software Engineering for Automated Vehicles: Addressing the Needs of Cars That Run on Software and Data, in: 2019 IEEE/ACM 41st International Conference on Software Engineering: Companion Proceedings (ICSE-Companion), IEEE. pp. 6–8. URL: https://ieeexplore...

    2019

  18. [24]

    Automating Safety Argument Change Impact Analysis for Machine Learning Com- ponents, in: 2022 IEEE 27th Pacific Rim International Symposium on Dependable Computing (PRDC), pp

    Cˆ arlan, C., Gauerhof, L., Gallina, B., Burton, S., 2022. Automating Safety Argument Change Impact Analysis for Machine Learning Com- ponents, in: 2022 IEEE 27th Pacific Rim International Symposium on Dependable Computing (PRDC), pp. 43–53. doi:10.1109/PRDC55274. 2022.00019

    2022

  19. [25]

    and Larsen, Rick,

    DeFazio, Peter A. and Larsen, Rick, . Final Committee Re- port—The Design, Development & Certification of the Boeing 737 MAX. The US House Committee on Transportation and Infras- tructure, Subcommittee on Aviation. Washington, DC: US House Committee on Transportation and Infra...

    2020

  20. [26]

    Trustworthy and collaborative traceability management: Experts’ feed- back on a blockchain-enabled framework

    Demi, S., S´ anchez-Gord´ on, M., Kristiansen, M., Larrucea, X., 2024. Trustworthy and collaborative traceability management: Experts’ feed- back on a blockchain-enabled framework. Journal of Software: Evolu- tion and Process , e2707

    2024

  21. [27]

    SafeOps: A Concept of Continuous Safety, in: 2020 16th European Dependable Computing Conference (EDCC), IEEE

    Fayollas, C., Bonnin, H., Flebus, O., 2020. SafeOps: A Concept of Continuous Safety, in: 2020 16th European Dependable Computing Conference (EDCC), IEEE. pp. 65–68. URL: https://ieeexplore. ieee.org/abstract/document/9237051/

    2020

  22. [28]

    A Matter of Trust: Ford’s Approach to Developing Self-Driving Vehicles

    Ford, . A Matter of Trust: Ford’s Approach to Developing Self-Driving Vehicles. URL: https://media.ford.com/content/dam/fordmedia/ North%20America/US/2021/06/17/ford-safety-report.pdf . ac- cessed: 2025-02-28

    2021

  23. [29]

    Validation of software tools according to ISO 26262 method 1c using continuous practices: applied to the software product TEST-GUIDE

    Fuchs, M., 2024. Validation of software tools according to ISO 26262 method 1c using continuous practices: applied to the software product TEST-GUIDE. Ph.D. thesis. Technische Hochschule Ingolstadt

    2024

  24. [30]

    Safety and Security of Cyber-Physical Systems: Engineering Dependable Software Using Principle-based Development

    Furrer, F.J., 2022. Safety and Security of Cyber-Physical Systems: Engineering Dependable Software Using Principle-based Development. Springer Nature

    2022

  25. [31]

    Vv methods safety assurance position paper

    Galbas, R., Nolte, M., Eberle, U., Hungar, H., Mosebach, H.H., Salem, N.F., Schittenhelm, H., Reich, J., Kirschbaum, T., Westhofen, L., 2024. Vv methods safety assurance position paper . 41

    2024

  26. [32]

    Gautham, S., Bakirtzis, G., Will, A., Jayakumar, A.V., Elks, C.R.,

  27. [33]

    The Automotive Take on Continuous Experimentation: A Multiple Case Study, in: 2019 45th Euromicro Conference on Software Engineering and Advanced Applica- tions (SEAA), IEEE

    Giaimo, F., Andrade, H., Berger, C., 2019. The Automotive Take on Continuous Experimentation: A Multiple Case Study, in: 2019 45th Euromicro Conference on Software Engineering and Advanced Applica- tions (SEAA), IEEE. pp. 126–130. URL: https://ieeexplore.ieee. org/abstract/doc...

    2019

  28. [34]

    Self-driving safety report

    GM, . Self-driving safety report. URL: https://www.gm.com/ stories/self-driving-cars. accessed: 2025-02-28

    2025

  29. [35]

    MLOps: Continuous delivery and automation pipelines in machine learning

    Google, a. MLOps: Continuous delivery and automation pipelines in machine learning. URL: https://cloud.google.com/architecture/ mlops-continuous-delivery-and-automation-pipelines-in-machine-learning . accessed: 2024-09-01

    2024

  30. [36]

    What is DevOps? URL: https://cloud.google.com/ devops

    Google, b. What is DevOps? URL: https://cloud.google.com/ devops. accessed: 2024-09-01

    2024

  31. [37]

    Review on self-driving cars using neural network architectures

    Gudla, R., Telidevulapalli, V.S., Kota, J.S., Mandha, G., et al., 2022. Review on self-driving cars using neural network architectures. World Journal of Advanced Research and Reviews 16, 736–746

    2022

  32. [38]

    Con- trolling Risk for Highly Automated Transportation Systems Operating in Complex Open Environments: A white paper of the SafeTRANS Closing the Gap Initiative

    Haas, L., Haider, A., Jung, R., Poguntke, T., Schick, B., Schneider, S.A., Wagner, F., Zeh, T., Benedikt, M., B¨ ode, E., et al., 2024. Con- trolling Risk for Highly Automated Transportation Systems Operating in Complex Open Environments: A white paper of the SafeTRANS Closing...

    2024

  33. [39]

    Requirements and software engineering for automotive perception systems: an interview study

    Habibullah, K.M., Heyn, H.M., Gay, G., Horkoff, J., Knauss, E., Borg, M., Knauss, A., Sivencrona, H., Li, P.J., 2024. Requirements and software engineering for automotive perception systems: an interview study. Requirements Engineering , 1–24

    2024

  34. [40]

    Integrating scenario- and contract-based verification for automated vessels

    Hake, G., Reiher, D., Mentjes, J., Hahn, A., 2024. Integrating scenario- and contract-based verification for automated vessels. Journal of Ma- rine Science and Technology , 1–21. 42

    2024

  35. [41]

    SafeScrum®-Agile development of safety-critical software

    Hanssen, G.K., St ˚ alhane, T., Myklebust, T., 2018. SafeScrum®-Agile development of safety-critical software. Springer

    2018

  36. [42]

    Improving 3d object detection for autonomous driving–a case study of data-driven development, in: International ATZ Conference, Springer

    Hartmannsgruber, A., Pinke, C., Jing, C., Pangottil, S., Grewe, R., Strauss, M., Mottok, J., Rojas, R., 2024. Improving 3d object detection for autonomous driving–a case study of data-driven development, in: International ATZ Conference, Springer. pp. 77–91

    2024

  37. [43]

    An empirical investigation of challenges of specifying training data and runtime monitors for critical software with machine learning and their relation to architectural decisions

    Heyn, H.M., Knauss, E., Malleswaran, I., Dinakaran, S., 2024. An empirical investigation of challenges of specifying training data and runtime monitors for critical software with machine learning and their relation to architectural decisions. Requirements Engineering 29, 97– 117

    2024

  38. [44]

    A review of testing object- based environment perception for safe automated driving

    Hoss, M., Scholtes, M., Eckstein, L., 2022. A review of testing object- based environment perception for safe automated driving. Automotive Innovation 5, 223–250

    2022

  39. [45]

    IKE Safety Report

    IKE, . IKE Safety Report. URL: https://global-uploads.webflow. com/5d5c3db08993c73c35421c12/5d9e954a4a6e6c7c7b0804d4_ Ike-SafetyReport-AC-03.pdf#page=8.34 . accessed: 2025-02-28

    2025

  40. [46]

    IEC 61508: Functional safety of electrical/electronic/programmable electronic safety-related systems

    International Electrotechnical Commission (IEC), 2010. IEC 61508: Functional safety of electrical/electronic/programmable electronic safety-related systems. Available online: https://webstore.iec.ch/ publication/22273

    2010

  41. [47]

    ISO/SAE 21434:2021, Road Vehicles — Cybersecu- rity Engineering

    ISO 21434, 2021. ISO/SAE 21434:2021, Road Vehicles — Cybersecu- rity Engineering. Standard. International Organization for Standard- ization

    2021

  42. [48]

    ISO 21448, Road vehicles — Safety of the intended functionality

    ISO 21448, 2022. ISO 21448, Road vehicles — Safety of the intended functionality. Standard. International Organization for Standardiza- tion

    2022

  43. [49]

    ISO 26262:2018 (all parts), Road vehicles — Functional safety

    ISO 26262:2018, 2018. ISO 26262:2018 (all parts), Road vehicles — Functional safety. Standard. International Organization for Standard- ization. 43

    2018

  44. [50]

    ISO/TR 4804:2020, Road vehicles – Safety and cybersecurity for automated driving systems – Design, verifica- tion and validation methods

    ISO/TR 4804:2020, 2020. ISO/TR 4804:2020, Road vehicles – Safety and cybersecurity for automated driving systems – Design, verifica- tion and validation methods. Standard. International Organization for Standardization

    2020

  45. [51]

    Continuous Learning Approach to Safety Engineering, in: CARS - Critical Automotive applications: Robustness & Safety, Zaragoza, Spain

    Johansson, R., Koopman, P., 2022. Continuous Learning Approach to Safety Engineering, in: CARS - Critical Automotive applications: Robustness & Safety, Zaragoza, Spain. URL: https://hal.science/ hal-03782627

    2022

  46. [52]

    Johansson, R., Sivencrona, H., 2021. Developing SEooC -Original Concepts and Implications When Extending to ADS, in: CARS 2021 6th International Workshop on Critical Automotive Applications: Ro- bustness & Safety, M¨ unich, Germany. URL: https://hal.science/ hal-03366362

    2021

  47. [53]

    Towards an AI-driven busi- ness development framework: A multi-case study

    John, M.M., Olsson, H.H., Bosch, J., 2023. Towards an AI-driven busi- ness development framework: A multi-case study. Journal of Software: Evolution and Process 35, e2432

    2023

  48. [54]

    Guidelines for Performing Sys- tematic Literature Reviews in Software Engineering

    Kitchenham, B.A., Charters, S., 2007. Guidelines for Performing Sys- tematic Literature Reviews in Software Engineering. Technical Re- port EBSE 2007-001. Keele University and Durham University Joint Report. URL: https://www.elsevier.com/__data/promis_misc/ 525444systematicrev...

    2007

  49. [55]

    Kodiak Safety Report 2020

    Kodiak, . Kodiak Safety Report 2020. URL: https://kodiak.cdn. prismic.io/kodiak/1233c64e-7ecd-4af8-8bea-00c31b6ae959_ Kodiak-2020-Safety-Report.pdf . accessed: 2025-02-28

    2020

  50. [56]

    Ul 4600: What to include in an autonomous vehicle safety case

    Koopman, P., 2023. Ul 4600: What to include in an autonomous vehicle safety case. Computer 56, 101–104. doi: 10.1109/MC.2023.3236171

    2023

  51. [57]

    Anatomy of a robotaxi crash: Lessons from the cruise pedestrian dragging mishap

    Koopman, P., 2024. Anatomy of a robotaxi crash: Lessons from the cruise pedestrian dragging mishap. arXiv preprint arXiv:2402.06046

    2024 arXiv

  52. [58]

    Redefining safety for autonomous vehicles

    Koopman, P., Widen, W., 2024. Redefining safety for autonomous vehicles. arXiv:2404.16768

    2024 arXiv

  53. [59]

    Breaking the tyranny of net risk metrics for automated vehicle safety

    Koopman, P., Widen, W.H., 2023. Breaking the tyranny of net risk metrics for automated vehicle safety. Available at SSRN 4634179 . 44

    2023

  54. [60]

    Safe operations of unmanned systems for reconnaissance in complex environ- ments army technology objective (source ato), in: Unmanned Systems Technology XIII, SPIE

    Kott III, N.J., Wellfare, M., van Lierop, T.K., Mottern, E., 2011. Safe operations of unmanned systems for reconnaissance in complex environ- ments army technology objective (source ato), in: Unmanned Systems Technology XIII, SPIE. pp. 11–18

    2011

  55. [61]

    Architecture as a Backbone for Safe De- vOps in Automotive Systems, in: 2022 IEEE 25th International Con- ference on Intelligent Transportation Systems (ITSC), IEEE

    Kugele, S., Broy, M., 2022. Architecture as a Backbone for Safe De- vOps in Automotive Systems, in: 2022 IEEE 25th International Con- ference on Intelligent Transportation Systems (ITSC), IEEE. pp. 4145–

    2022

  56. [62]

    Data-Centric Communica- tion and Containerization for Future Automotive Software Architec- tures, in: 2018 IEEE International Conference on Software Architec- ture (ICSA), IEEE

    Kugele, S., Hettler, D., Peter, J., 2018. Data-Centric Communica- tion and Containerization for Future Automotive Software Architec- tures, in: 2018 IEEE International Conference on Software Architec- ture (ICSA), IEEE. pp. 65–6509

    2018

  57. [63]

    Report to the boards of directors of cruise LLC Gm cruise holdings LLC and general motors Holdings LLC regarding the October 2 2023 accident in San Francisco

    Lawyer, Q.E.T., 2024. Report to the boards of directors of cruise LLC Gm cruise holdings LLC and general motors Holdings LLC regarding the October 2 2023 accident in San Francisco

    2024

  58. [64]

    Lee, S.W., Choi, Y.G., Jeon, J.W., 2024. Enhancing Autonomous Driving Systems through ROS2 and A WS Cloud: V2I Interaction and HPC Data Processing, in: 2024 IEEE 33rd International Symposium on Industrial Electronics (ISIE), IEEE. pp. 1–6

    2024

  59. [65]

    Causa- lops—towards an industrial lifecycle for causal probabilistic graphical models

    Maier, R., Schlattl, A., Guess, T., Mottok, J., 2024. Causa- lops—towards an industrial lifecycle for causal probabilistic graphical models. Information and Software Technology 174, 107520

    2024

  60. [66]

    Mars Auto Safety Report

    Mars-Auto, . Mars Auto Safety Report. URL: https://marsauto. com/VSSA_safety_report.pdf. accessed: 2025-02-28

    2025

  61. [67]

    Introducing DRIVE PILOT: An Au- tomated Driving System for the Highway

    Mercedes-Benz, . Introducing DRIVE PILOT: An Au- tomated Driving System for the Highway. URL: https: //group.mercedes-benz.com/dokumente/innovation/sonstiges/ 2023-03-06-vssa-mercedes-benz-drive-pilot.pdf#page=42.07 . accessed: 2025-02-28

    2023

  62. [68]

    Meyers, B., Gadeyne, K., Oakes, B., Bernaerts, M., Vangheluwe, H., Denil, J., 2019. A Model-Driven Engineering Framework to Support 45 the Functional Safety Process, in: 2019 ACM/IEEE 22nd Interna- tional Conference on Model Driven Engineering Languages and Sys- tems Companion...

    2019

  63. [69]

    Continuous Verification of Automotive Software

    M¨ ortstrand, M., 2023. Continuous Verification of Automotive Software

    2023

  64. [70]

    DevOps and Safety? SafeOps! To- wards Ensuring Safety in Feature-Driven Development with Frequent Releases, in: Trapp, M., Schoitsch, E., Guiochet, J., Bitsch, F

    Munk, P., Schweizer, M., 2022. DevOps and Safety? SafeOps! To- wards Ensuring Safety in Feature-Driven Development with Frequent Releases, in: Trapp, M., Schoitsch, E., Guiochet, J., Bitsch, F. (Eds.), Computer Safety, Reliability, and Security. SAFECOMP 2022 Work- shops, Spri...

    2022

  65. [71]

    Myklebust, T., St ˚ alhane, T., Hanssen, G., 2020. Agile safety case and DevOps for the automotive industry, in: 30th Eu- ropean Safety and Reliability Conference and the 15th Proba- bilistic Safety Assessment and Management Conference. URL: https://www.researchgate.net/profil...

    2020

  66. [72]

    Agile safety case for vehi- cle trial operations

    Myklebusta, T., St ˚ alhaneb, T., Wuc, S., . Agile safety case for vehi- cle trial operations. Probabilistic Safety Assessment and Management PSAM 16

  67. [74]

    Nouri, A., Berger, C., T¨ orner, F., 2023. On STPA for Distributed Development of Safe Autonomous Driving: An Interview Study, in: Proceedings of the 49th EUROMICRO Conference on Software Engi- neering and Advanced Applications (SEAA), DURRES, ALBANIA

    2023

  68. [75]

    Nouri, A., Cabrero-Daniel, B., Torner, F., Sivencrona, H., Berger, C., 2024a. Welcome Your New AI Teammate: On Safety Anal- ysis by Leashing Large Language Models, in: Proceedings of the 46 IEEE/ACM 3rd International Conference on AI Engineering - Software Engineering for AI, ...

  69. [76]

    Engineering safety requirements for autonomous driving with large language models

    Nouri, A., Cabrero-Daniel, B., T¨ orner, F., Sivencrona, H., Berger, C., 2024b. Engineering safety requirements for autonomous driving with large language models. URL: https://arxiv.org/abs/2403.16289, arXiv:2403.16289

    arXiv

  70. [77]

    NVIDIA Self-Driving Safety Report

    NVIDIA, . NVIDIA Self-Driving Safety Report. URL: https://www. nvidia.com/en-us/self-driving-cars/ . accessed: 2025-02-28

    2025

  71. [78]

    Artificial Intelligence for Safety-Critical Systems in Indus- trial and Transportation Domains: A Survey

    Perez-Cerrolaza, J., Abella, J., Borg, M., Donzella, C., Cerquides, J., Cazorla, F.J., Englund, C., Tauber, M., Nikolakopoulos, G., Flores, J.L., 2024. Artificial Intelligence for Safety-Critical Systems in Indus- trial and Transportation Domains: A Survey. ACM Comput. Surv. 5...

    2024 doi

  72. [79]

    Raghupatruni, I., Karjee, S., Gupta, A., Naik, V., Huber, T.,

  73. [80]

    Rubasinghe, I., Meedeniya, D., Perera, I., 2018. Traceability Man- agement with Impact Analysis in DevOps based Software Devel- opment, in: 2018 International Conference on Advances in Com- puting, Communications and Informatics (ICACCI), pp. 1956–1962. doi:10.1109/ICACCI.2018.8554399

    2018

  74. [81]

    Security as cul- ture: a systematic literature review of devsecops, in: Proceedings of the IEEE/ACM 42nd international conference on software engineering workshops, pp

    S´ anchez-Gord´ on, M., Colomo-Palacios, R., 2020. Security as cul- ture: a systematic literature review of devsecops, in: Proceedings of the IEEE/ACM 42nd international conference on software engineering workshops, pp. 266–269

    2020

  75. [82]

    Con- tinuous Compliance in the Automotive Industry

    Santilli, T., Pelliccione, P., Wohlrab, R., Shahrokni, A., 2024. Con- tinuous Compliance in the Automotive Industry. IEEE Software 41, 134–142. doi: 10.1109/MS.2023.3342974. 47

    2024

  76. [83]

    Towards Continuous Safety Assurance for Autonomous Systems, in: 2022 6th International Conference on System Reliability and Safety (ICSRS), pp

    Schleiss, P., Carella, F., Kurzidem, I., 2022. Towards Continuous Safety Assurance for Autonomous Systems, in: 2022 6th International Conference on System Reliability and Safety (ICSRS), pp. 457–462. doi:10.1109/ICSRS56243.2022.10067323

    2022

  77. [84]

    ADS standardization landscape: Mak- ing sense of its status and of the associated research questions

    Schnelle, S., Favaro, F.M., 2023. ADS standardization landscape: Mak- ing sense of its status and of the associated research questions. arXiv preprint arXiv:2306.17682

    2023 arXiv

  78. [85]

    Synthetic data generation for the continuous development and testing of autonomous construction machinery

    Schuster, A., Hagmanns, R., Sonji, I., L¨ ocklin, A., Petereit, J., Ebert, C., Weyrich, M., 2023. Synthetic data generation for the continuous development and testing of autonomous construction machinery. at- Automatisierungstechnik 71, 953–968

    2023

  79. [86]

    Schwall, M., Daniel, T., Victor, T., Favaro, F., Hohnhold, H.,

  80. [87]

    Hid- den Technical Debt in Machine Learning Systems

    Sculley, D., Holt, G., Golovin, D., Davydov, E., Phillips, T., Ebner, D., Chaudhary, V., Young, M., Crespo, J.F., Dennison, D., 2015. Hid- den Technical Debt in Machine Learning Systems. Advances in neural information processing systems 28

    2015

  81. [89]

    Design and automatic generation of safety cases of ml-enabled autonomous driving systems

    Sivakumar, M., 2024. Design and automatic generation of safety cases of ml-enabled autonomous driving systems

    2024

  82. [90]

    Prompting GPT–4 to support automatic safety case generation

    Sivakumar, M., Belle, A.B., Shan, J., Shahandashti, K.K., 2024. Prompting GPT–4 to support automatic safety case generation. Expert Systems with Applications 255, 124653

    2024

  83. [91]

    arXiv preprint arXiv:2011.00038

    Waymo public road safety performance data. arXiv preprint arXiv:2011.00038

    2011 arXiv

  84. [92]

    Trustworthiness assurance assessment for high-risk ai-based systems

    Stettinger, G., Weissensteiner, P., Khastgir, S., 2024. Trustworthiness assurance assessment for high-risk ai-based systems. IEEE Access

    2024

  85. [93]

    Application of Model- Based Systems Engineering Methods in Virtual Homologation Pro- 48 cedures for Automated Driving Functions

    Temmen, T., Meyer, M., Wachtmeister, L., Zabihi, M., Kugler, C., Christiaens, S., Rumpe, B., Andert, J., 2024. Application of Model- Based Systems Engineering Methods in Virtual Homologation Pro- 48 cedures for Automated Driving Functions. pp. 1–14. doi: 10.1007/ 978-3-658-45196-7_1

    2024

  86. [94]

    Toward a hybrid causal framework for autonomous vehicle safety analysis

    Thomas, S., Groth, K.M., 2023. Toward a hybrid causal framework for autonomous vehicle safety analysis. Proceedings of the Institution of Mechanical Engineers, Part O: Journal of Risk and Reliability 237, 367–388

    2023

  87. [95]

    Migration of Model-based Software Components to Service-oriented Architectures

    Tomar, G., 2024. Migration of Model-based Software Components to Service-oriented Architectures. ATZelectronics worldwide 19, 50–54

    2024

  88. [96]

    ISTQB ® Certified Tester Foundation Level

    Stapp, L., Roman, A., Pilaeten, M., 2024. ISTQB ® Certified Tester Foundation Level. Springer

    2024

  89. [97]

    TUSIMPLE SAFETY REPORT

    TUSIMPLE, . TUSIMPLE SAFETY REPORT. URL: https://www.tusimple.com/wp-content/uploads/2021/03/ TuSimple-Safety-Report.pdf. accessed: 2025-02-28

    2021

  90. [98]

    UN Regulation No

    UNECE, 2021. UN Regulation No. 157 - Auto- mated Lane Keeping Systems (ALKS). URL: https: //unece.org/transport/documents/2021/03/standards/ un-regulation-no-157-automated-lane-keeping-systems-alks

    2021

  91. [99]

    Department of Transportation NHTSA, 2017

    U.S. Department of Transportation NHTSA, 2017. A Vision for Safety 2.0. Technical Report. U.S. Department of Transportation, National Highway Traffic Safety Administration

    2017

  92. [100]

    Volkswagen’s Safety Self-Assessment for Automated Mobility Services

    Volkswagen, . Volkswagen’s Safety Self-Assessment for Automated Mobility Services. URL: https://media.vw.com/assets/documents/ original/17321-VolkswagenVSSAV12023.pdf. accessed: 2025-02-28

    2025

  93. [101]

    Torc Safety Report

    Torc-Robotics, . Torc Safety Report. URL: https://torc.ai/ wp-content/uploads/2024/06/Torc-Robotics-VSSA.pdf . accessed: 2025-02-28

    2024

  94. [102]

    Wardzi´ nski, A., Jarzebowicz, A., 2023. Development of the System Assurance Reference Model for Generating Modular Assurance Cases, in: 2023 IEEE 28th Pacific Rim International Symposium on Depend- able Computing (PRDC), pp. 99–110. doi: 10.1109/PRDC59308.2023. 00022. 49

    2023

  95. [103]

    Continuous De- ployment for Dependable Systems with Continuous Assurance Cases, in: 2019 IEEE International Symposium on Software Reliability En- gineering Workshops (ISSREW), IEEE

    Warg, F., Blom, H., Borg, J., Johansson, R., 2019. Continuous De- ployment for Dependable Systems with Continuous Assurance Cases, in: 2019 IEEE International Symposium on Software Reliability En- gineering Workshops (ISSREW), IEEE. pp. 318–325. URL: https: //ieeexplore.ieee.o...

    2019

  96. [104]

    Salience4cav public report: Safety lifecycle enabling continuous deployment for connected automated vehicles

    Warg, F., Thors´ en, A., Chen, D., Henriksson, J., Rodrigues de Cam- pos, G., 2024. Salience4cav public report: Safety lifecycle enabling continuous deployment for connected automated vehicles

    2024

  97. [105]

    Waymo safety report

    Waymo, . Waymo safety report. URL:https://waymo.com/research/ waymo-safety-report/. accessed: 2025-02-28

    2025

  98. [106]

    Waabi’s Voluntary Safety Self-Assessment

    Waabi, . Waabi’s Voluntary Safety Self-Assessment. URL: https: //waabi.ai/vssa/. accessed: 2025-02-28

    2025

  99. [107]

    Weiss, G., Zeller, M., Schoenhaar, H., Drabek, C., Kreutz, A., 2024. Approach for Argumenting Safety on Basis of an Operational Design Domain, in: Proceedings of the IEEE/ACM 3rd International Confer- ence on AI Engineering-Software Engineering for AI, pp. 184–193

    2024

  100. [108]

    Toward a safe MLOps process for the continuous development and safety as- surance of ML-based systems in the railway domain

    Zeller, M., Waschulzik, T., Schmid, R., Bahlmann, C., 2024. Toward a safe MLOps process for the continuous development and safety as- surance of ML-based systems in the railway domain. AI and Ethics 4, 123–130

    2024

  101. [109]

    SAFETY INNOV ATION AT ZOOX

    ZOOX, . SAFETY INNOV ATION AT ZOOX. URL: https://www.datocms-assets.com/106048/1696537818-zoox_ safety_report_volume1_2018.pdf. accessed: 2025-02-28. 50

    2025

  102. [111]

    Waymo’s safety methodologies and safety readiness determinations

    Webb, N., Smith, D., Ludwick, C., Victor, T., Hommes, Q., Favaro, F., Ivanov, G., Daniel, T., 2020. Waymo’s safety methodologies and safety readiness determinations. arXiv preprint arXiv:2011.00054

    2020 arXiv

  103. [2020]

    UP2DATE: Safe and secure over-the-air software updates on high-performance mixed-criticality systems, in: 2020 23rd Euromicro Conference on Digital System Design (DSD), pp. 344–351. doi: 10. 1109/DSD51259.2020.00063

    2020

  104. [2021]

    Technical Report

    Towards Establishing Continuous-X Pipeline Using Mod- ular Software-in-the-Loop Test Environments. Technical Report. SAE Technical Paper. URL: https://www.sae.org/publications/ technical-papers/content/2021-26-0412/

    2021

  105. [2022]

    (Eds.), Computer Safety, Reliability, and Security, Springer Interna- tional Publishing, Cham

    STPA-Driven Multilevel Runtime Monitoring for In-Time Haz- ard Detection, in: Trapp, M., Saglietti, F., Spisl¨ ander, M., Bitsch, F. (Eds.), Computer Safety, Reliability, and Security, Springer Interna- tional Publishing, Cham. pp. 158–172

  106. [4150]

    URL: https://ieeexplore.ieee.org/abstract/document/ 9922097/

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.