Pith. sign in

REVIEW 4 major objections 5 minor 74 references

Reliability, Resilience and Human Factors Engineering for Trustworthy AI Systems

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper proposes a human-centric framework, HC-AIRM, that applies reliability and resilience engineering to AI systems so that failures become countable, recoverable, and improvable across the AI lifecycle.

desk verdict Useful conceptual framing for AI reliability metrics, but the OpenAI case study has a unit error and the discrete-event assumption is unproven. read the letter →

arxiv 2411.08981 v1 pith:F3XQS46Z submitted 2024-11-13 cs.AI cs.SYeess.SY

Saurabh Mishra , Anand Rao , Ramayya Krishnan , Bilal Ayyub , Amin Aria , Enrico Zio This is my paper · ORCID
classification cs.AIcs.SYeess.SY
keywords AIreliabilityresilienceengineeringhumanfactorsprognosticsandhealthmanagementMTBFbathtubcurveanalysistrustworthy
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Reliability engineering has kept bridges, aircraft, and power plants safe for decades; this paper asks whether the same toolbox can keep AI systems trustworthy. Its central proposal is the Human-Centric AI Reliability Model (HC-AIRM), which treats an AI system as five interacting subsystems—data, model, computing infrastructure, code and software, and human interaction—and applies reliability engineering before deployment to prevent failures and resilience engineering after deployment to recover from them. Underpinning both is prognostics and health management, the practice of continuously monitoring a system to catch problems before they become outages. To show the framework is practical, the authors apply it to public system-status incident data from OpenAI, computing component-level failure rates, mean time between failures, and recovery metrics that reveal an infant-mortality pattern in one component. If the framework works, the payoff is a quantitative and human-aware language for AI safety that regulators, insurers, and engineering teams can share.

What carries the argument

The central object is the Human-Centric AI Reliability Model (HC-AIRM), a lifecycle framework that makes human factors a first-class reliability variable. It decomposes AI systems into data, model, cloud and computing infrastructure, code and software, and human subsystems, then maps each to pre-deployment reliability (design and development, using structured failure-mode analysis and human reliability analysis) and post-deployment resilience (operation, using human-in-the-loop testing, situational awareness analysis, and continuous monitoring). The quantitative machinery is a set of borrowed reliability functions and metrics—the reliability function, hazard rate, MTBF, failure rate, mean time to data drift, cost of downtime, and probability of failure on demand—plus resilience formulas such as the AI Resilience Index (recovery rate divided by frequency of failures) and the bathtub curve, which the paper adapts to show early infant-mortality failures, random operational failures, and wear-out failures driven by human interaction patterns. This machinery carries the argument by turning vague notions of AI trustworthiness into countable, monitorable events.

What would settle it

A concrete test would be to take the same OpenAI status-page incidents and have several independent teams classify each incident into the paper's five subsystems using a written rubric; if inter-rater agreement is low, or if reclassification substantially changes the MTBF and failure-rate ranking in Table 7 and the infant-mortality trend in Figure 9, the framework's quantitative conclusions do not survive. A second, sharper check would be to run the same counting method on another AI platform's status data and see whether the bathtub-curve pattern of high early failure rates followed by stabilization appears reliably, as predicted.

Watch

Extended reading notes

Core claim

The paper's central claim is that AI trustworthiness can be engineered, not just inspected, by integrating reliability engineering, resilience engineering, human factors engineering, and prognostics and health management into the AI lifecycle. The authors define an AI system as a repairable, better-than-new system of five subsystems—data, model, computing infrastructure, code and software, and human interaction—and argue that every subsystem has identifiable failure modes with measurable metrics. They introduce the HC-AIRM framework to embed human reliability analysis into the design, development, deployment, and operation phases, and they propose quantitative measures such as failure rate, MTBF, mean time to data drift, cost of downtime, and an AI Resilience Index. The case study on OpenAI status incidents from May to October 2024 demonstrates the framework by reverse-engineering incident reports into subsystem and component failures, computing MTBF and recovery metrics, and identifying an infant-mortality failure-rate pattern in the ChatGPT component. The conclusion offered is that this engineering vocabulary makes AI failures manageable and creates a bridge to policy, regulation, and insurance.

Load-bearing premise

The load-bearing premise is that AI failures can be counted and labeled as discrete events with clear start times, end times, and subsystem causes, so that classic reliability metrics like mean time between failures keep their meaning when applied to an AI system.

Editorial extensions

If this is right

  • Engineering teams can use component-level MTBF, mean time to recovery, and failure-rate dashboards to prioritize reliability work on the subsystems that fail most often.
  • Pre-deployment human reliability analysis, such as human error probability assessment and cognitive work analysis, can catch design and labeling errors before release and reduce early infant-mortality failures.
  • Post-deployment resilience metrics, such as recovery rate divided by failure frequency and time to recovery, allow operators to measure how well a deployed AI system bounces back from disruptions and to compare recovery strategies.
  • The better-than-new repairable system view reframes AI updates: each version release is a repair event, and reliability should be tracked version by version rather than only at initial deployment.
  • The same framework can produce quantitative inputs for AI insurance, return-on-investment analysis, and policy decisions by attaching costs and probabilities to specific failure modes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the counting assumption holds, a natural extension is a standardized public incident-reporting taxonomy for AI platforms, so that MTBF and failure rates become comparable across providers and regulators can benchmark reliability the way safety statistics are benchmarked in aviation or nuclear power.
  • The framework implicitly predicts that the bathtub curve's infant-mortality phase is not unique to OpenAI; newly released AI features and models should generally show higher failure rates in their first weeks, followed by stabilization, across other platform status pages.
  • Because the paper treats human interaction as a subsystem with its own failure modes, an implication left implicit is that user-interface design and user training are reliability interventions, not just usability concerns, so usability testing could be reframed as a form of reliability testing.
  • The reverse-engineering of status incidents into subsystems could be automated and validated: a classifier trained on a labeled incident corpus could assign failure modes and subsystems, providing a large-scale test of whether the taxonomy is robust.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a framework, HC-AIRM, that integrates reliability engineering, resilience engineering, human factors engineering, and prognostics and health management (PHM) across the AI system lifecycle. It decomposes AI systems into Data, Model, Computing Infrastructure, Code+Software, and Human subsystems, adapts classical reliability metrics such as MTBF, failure rate, MTTD, POFOD, and bathtub-curve analysis, introduces an AI Resilience Index, and illustrates the framework with a case study built from OpenAI status-page incidents between May 1 and October 21, 2024. The authors explicitly frame the work as a conceptual and methodological integration and as a research agenda rather than a definitive empirical validation.

Significance. If the central claims were validated, the paper would provide a useful bridge between established engineering reliability practice and AI governance, and its mapping to OECD, NIST, and EU policy frameworks increases its relevance for standards and regulation. The paper's strengths include the consistent use of standard reliability definitions, clear citation of the underlying resilience metric from Ayyub [5], and an honest acknowledgment of the case study's subjective and data-constrained nature. The mathematical definitions in Sections 3 and 4 are conventional and correctly cited. However, the paper ships no machine-checked proofs, reproducible analysis code, or parameter-free derivations, and the empirical demonstration currently contains a decisive internal inconsistency that undermines the quantitative claims.

major comments (4)
  1. [§6.1, Table 7] Table 7 is internally inconsistent: for the ChatGPT row, MTBF = 10 days implies a constant failure rate of approximately 0.1 per day, yet the table reports a Failure Rate of 9.8 per day; the other rows show the same factor-of-100 discrepancy (e.g., Authentication: MTBF 22 days, failure rate 4.6/day). As printed, the failure-rate column appears to be 100/MTBF rather than 1/MTBF, so the reliability metrics and the "infant mortality" reading of Figure 9 are not supported by the reported numbers.
  2. [§1, §6.1, Appendix C] The framework's core assumption that AI failures are discrete, identifiable, countable, isolatable, and reproducible events (Appendix C) is never operationalized for the Model subsystem. Section 1 defines failure as "any deviation from expected performance," which for generative AI includes continuous, context-dependent output-quality problems such as hallucinations or bias; no procedure is given for converting such deviations into discrete restorable failures. The OpenAI status-page case study concerns infrastructure outages and therefore does not validate the application of MTBF, failure rate, or bathtub-curve analysis to the model-output failures that are central to the paper's stated scope.
  3. [§6, Table 6, Figure 9] The empirical demonstration is based on manual reverse-engineering of status-page incidents and is presented without uncertainty quantification: Table 6 does not define the Severity, Occurrence, and Detection scales used in the RPN or the Impact Score, and Figure 9's logarithmic trendline is shown without fit statistics or error bars. Because the paper's central claim includes demonstrating the framework's practical applicability, these analyses need an inter-rater reliability assessment or a clearly labeled illustrative status.
  4. [§4, Eq. (9)] Equation (9), the resilience metric, is not self-contained: the symbols F and R appearing in the numerator are not defined in the text or in a notation list, and Eq. (10) does not connect Pfail and Prec to these symbols. Without definitions (or an explicit reference to the definitions in Ayyub [5] with the required notation), the metric cannot be applied or audited.
minor comments (5)
  1. [Abstract] The abstract contains typos ("an integrate framework" should be "an integrated framework") and inconsistent capitalization of "OpenAI."
  2. [§2.2] There is a duplicated passage: the sentence beginning "moteraction between AI systems and their environment..." appears twice nearly verbatim in the same subsection; one copy should be deleted.
  3. [Table 3] In the Compute subsystem row, "BFBF (Mean Time Between Failures)" should read "MTBF (Mean Time Between Failures)."
  4. [§6.1] The text refers to "Table 8b" for the component-level breakdown, but the breakdown is Figure 8(b), not a table; the cross-reference should be corrected.
  5. [Appendix E] The incident tables appear mis-numbered (the incidents sample is labeled Table 8 rather than Table 12), and Tables 10 and 11 are identical duplicates; renumber and deduplicate.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the paper integrates standard reliability definitions with an explicitly acknowledged illustrative case study; only minor non-load-bearing self-citations are present.

full rationale

The paper does not derive a prediction from fitted inputs. Section 3 states textbook reliability definitions (R(t)=P(T>t), h(t)=f(t)/R(t), MTBF) as inherited from reliability theory [3,73]; these are self-contained definitions, not outputs of the paper's framework. Section 4's resilience metric (Eq. 9) is explicitly cited to Ayyub [5], a standard external metric (and co-author), and is used as an adaptation rather than as evidence for the paper's central claim. The HC-AIRM model in Section 5.5 is a synthesis of named HRA techniques (FMEA, HEP, CWA, HITL), not a derivation whose conclusion equals its premise. The case study in Section 6.1 computes descriptive MTBF/MTTR/failure-rate numbers from public OpenAI status incidents and is explicitly labeled 'subjective analysis' with 'constrained data availability'; no fitted parameter is renamed as a prediction, and no equation reduces to its own input. The only notable self-citations (Ayyub [5]; Rao [56]) are not load-bearing, since the cited resilience formula and operationalization concept are externally published. A separate correctness concern, not circularity, is that Table 7 lists ChatGPT MTBF=10 days with failure rate 9.8/day, which is arithmetically inconsistent unless the rate is per 100 days; this does not affect the circularity verdict.

Assumptions & free parameters 3 free parameters · 5 assumptions · 3 invented entities

The central claim rests on transferring reliability theory, bathtub curve reasoning, infrastructure resilience formulas, and nuclear/aerospace human reliability analysis to AI systems by analogy. The case study adds manual incident classification and hand-assigned risk scores, which are free choices not independently auditable. No new entity with external falsifiable evidence is introduced.

free parameters (3)
  • OpenAI incident-to-subsystem classification = not quantified; manual assignment
    Figure 8 and Section 6.1 assign each status incident to subsystems (Code+Software, Computing, Data, Human, Model) without a published rubric or inter-rater agreement; counts would shift with the classifier's judgment.
  • Table 6 Impact Score and RPN components = raw Severity, Occurrence, Detection not shown
    Risk Priority Numbers in Table 6 are products of hand-assigned scores; only the products are reported, so the input values are free choices that determine the prioritization.
  • Logarithmic trendline in Figure 9 = not stated
    The yellow trendline is fit to the computed failure-rate series and used to suggest 'infant mortality' and 'random failure' phases; no fit parameters or goodness-of-fit are given.
assumptions (5)
  • standard math Classical reliability definitions (R(t), f(t), h(t)) transfer to AI system components.
    Section 3 inherits Eqs. (1)-(4) from classical reliability theory without AI-specific justification beyond analogy.
  • domain assumption AI failure events are discrete, countable, and independently identifiable.
    Section 1 defines failure as 'any deviation from expected performance'; Sections 3 and 6.1 treat status-page outages and output-quality problems as comparable countable events.
  • domain assumption Component reliabilities multiply independently in Eq. (5).
    Eq. (5) multiplies R_data, R_model, R_compute, R_software, and R_human; the authors acknowledge this assumes independence, which may not hold for highly integrated AI systems.
  • domain assumption The bathtub curve applies to the AI system lifecycle.
    Section 3 and Appendix A map AI phases to infant-mortality, random-failure, and wear-out phases; Figure 9's annotation is called 'purely suggestive' by the authors.
  • domain assumption Infrastructure resilience formulas transfer to AI systems.
    Section 4 imports the resilience metric from Ayyub [5] via analogy to physical infrastructure, with no AI-specific validation.
invented entities (3)
  • Human-Centric AI Reliability Model (HC-AIRM)
    purpose: Conceptual framework integrating HRA, FMEA, HEP, and CWA into AI lifecycle phases to improve trustworthiness.
    Introduced in Section 5.5 as a novel framework; no quantitative predictions, controlled experiments, or external validation are provided.
  • AI Resilience Index (ARI)
    purpose: Metric to quantify AI system resilience as recovery rate divided by failure frequency.
    Defined in Eq. (11) as a ratio of two empirical quantities; no demonstration that it tracks safety or trustworthiness is given.
  • Mean Time to Data Drift (MTTD)
    purpose: Metric for expected time until input data distribution shifts enough to degrade model performance.
    Defined in Eq. (6); operationalization relies on drift-detection choices such as KL divergence or KS tests, and no empirical estimate is reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Reliability, Resilience and Human Factors Engineering for Trustworthy AI Systems." pith.science (2026). https://pith.science/paper/F3XQS46Z

@misc{pith2026241108981,
  author       = {Pith},
  title        = {Pith review of: Reliability, Resilience and Human Factors Engineering for Trustworthy AI Systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/F3XQS46Z}},
  note         = {Machine review of arXiv:2411.08981}
}
read the original abstract

As AI systems become integral to critical operations across industries and services, ensuring their reliability and safety is essential. We offer a framework that integrates established reliability and resilience engineering principles into AI systems. By applying traditional metrics such as failure rate and Mean Time Between Failures (MTBF) along with resilience engineering and human reliability analysis, we propose an integrate framework to manage AI system performance, and prevent or efficiently recover from failures. Our work adapts classical engineering methods to AI systems and outlines a research agenda for future technical studies. We apply our framework to a real-world AI system, using system status data from platforms such as openAI, to demonstrate its practical applicability. This framework aligns with emerging global standards and regulatory frameworks, providing a methodology to enhance the trustworthiness of AI systems. Our aim is to guide policy, regulation, and the development of reliable, safe, and adaptable AI technologies capable of consistent performance in real-world environments.

Figures

Figures reproduced from arXiv: 2411.08981 by the authors.

Figure 1
Figure 1. Illustrating AI Failure Modes: High-Level Taxonomy and Relating Sector-Impact-Failure Modes [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Abstraction of AI System, a simple view. Expanded view available in 4. [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. AI System lifecycle stages and the reliability-resilience framework for trustworthy AI. AI Lifecycle - [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (10 more)
Figure 5
Figure 5. Figure 5: Proposed resilience metrics for an AI system. [PITH_FULL_IMAGE:figures/full_fig_p022_5.png]
Figure 6
Figure 6. Figure 6: Adapted Bathtub Curve Illustrating Human Factors in AI System Failures. The Figure adapts the traditional [PITH_FULL_IMAGE:figures/full_fig_p024_6.png]
Figure 7
Figure 7. Figure 7: Human-Centric AI Reliability Model (HC-AIRM) Framework [PITH_FULL_IMAGE:figures/full_fig_p025_7.png]
Figure 8
Figure 8. Figure 8: Failure Events by Components (Modules), OpenAI Status, over a sample period between May 1, 2024, and [PITH_FULL_IMAGE:figures/full_fig_p027_8.png]
Figure 9
Figure 9. Figure 9: Failure Rate for ChatGPT Component, May 1 2024 - October 21, 2024. The grey line is calculated failure [PITH_FULL_IMAGE:figures/full_fig_p028_9.png]
Figure 10
Figure 10. Figure 10: Risk Rates and Failure Times in AI Systems [PITH_FULL_IMAGE:figures/full_fig_p035_10.png]
Figure 11
Figure 11. Figure 11: Illustration of stress-life relationship obtained from time-dependent reliability analysis [35]. [PITH_FULL_IMAGE:figures/full_fig_p040_11.png]
Figure 12
Figure 12. Figure 12: Histogram of Time Between Failures (TBF) for AI Incidents: Distribution of Days Between Reported [PITH_FULL_IMAGE:figures/full_fig_p043_12.png]
Figure 13
Figure 13. Figure 13: Mean Time Between Failures (MTBF) Over Time [PITH_FULL_IMAGE:figures/full_fig_p043_13.png]
Figure 14
Figure 14. Figure 14: AI System Failure Modes Distribution Source: AI Incident Database 43 [PITH_FULL_IMAGE:figures/full_fig_p043_14.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

74 extracted references · 69 canonical work pages

  1. [5]

    Bilal M. Ayyub. Systems resilience for multihazard environments: Definition, metrics, and valuation for decision making. Risk Analysis, 34(2):340–355, 2014

    work page 2014

  2. [1]

    Al-Khalifa, and Abdel Magid Hamouda

    Abdulaziz Aldoseri, Khalifa N. Al-Khalifa, and Abdel Magid Hamouda. Re-thinking data strategy and integration for artificial intelligence: Concepts, opportunities, and challenges. Appl. Sci., 13(12):7082, 2023. Submission received: 3 May 2023 / Revised: 30 May 2023 / Accepted: 7 June 2023 / Published: 13 June 2023

    work page 2023

  3. [2]

    AI Oversight and Human Mistakes: Evidence from Centre Court

    David Almog, Romain Gauriot, Lionel Page, and Daniel Martin. Ai oversight and human mistakes: Evidence from centre court. arXiv preprint arXiv:2401.16754, 2024. February 18, 2024

    work page Pith review arXiv 2024

  4. [3]

    Uncertainty in Risk Assessment: The Representation and Treatment of Uncertainties by Probabilistic and Non-Probabilistic Methods

    Terje Aven, Enrico Zio, Piero Baraldi, and Roger Flage. Uncertainty in Risk Assessment: The Representation and Treatment of Uncertainties by Probabilistic and Non-Probabilistic Methods. Wiley Publishing, 1st edition, 2014

    work page 2014

  5. [4]

    AI systems trustworthiness assessment: State of the art

    Afef Awadid, Kahina Amokrane-Ferka, Henri Sohier, Juliette Mattioli, Faouzi Adjed, Martin Gonzalez, and Souhaiel Khalfaoui. AI systems trustworthiness assessment: State of the art. Workshop on Model-based System Engineering and Artificial Intelligence - MBSE-AI Integration, 2024

    work page 2024

  6. [6]

    Kilr Ayyub, Bilal M

    G. Kilr Ayyub, Bilal M. Practical resilience metrics for planning, design, and decision making. ASCE-ASME Journal of Risk and Uncertainty in Engineering Systems Part A Civil Engineering 1(3):04015008, May 2015

    work page 2015

  7. [7]

    Andrea Bajcsy and Jaime F. Fisac. Human–AI safety: A descendant of generative AI and control systems safety. arXiv:2405.09794v1 [cs.AI], May 2024. License: arXiv.org perpetual non-exclusive license

    arXiv 2024

  8. [8]

    The human error in algorithms and its implications for human rights

    Veronica Barassi. The human error in algorithms and its implications for human rights. White Paper on Artificial Intelligence - A European Approach, 2020. In press, European Union Commission Report

    work page 2020

Show all 74 references
  1. [9]

    Bastani and I.-R

    F. Bastani and I.-R. Chen. Assessment of the reliability of AI programs, 1990

    1990

  2. [10]

    Probabilistic Risk Analysis: Foundations and Methods

    Tim Bedford and Roger Cooke. Probabilistic Risk Analysis: Foundations and Methods. Cambridge University Press, 2001

    2001

  3. [11]

    Reliability Engineering

    Alessandro Birolini. Reliability Engineering. Springer, 2023

    2023

  4. [12]

    Dynamic adversarial attacks on autonomous driving systems

    Amirhosein Chahe, Chenan Wang, Abhishek Jeyapratap, Kaidi Xu, and Lifeng Zhou. Dynamic adversarial attacks on autonomous driving systems. arXiv preprint arXiv:2312.06701, 2023

    2023 arXiv

  5. [13]

    AI system development: Cnil’s recommendations to comply with the gdpr

    CNIL. AI system development: Cnil’s recommendations to comply with the gdpr. https://www.cnil.fr/en/ ai-system-development-cnils-recommendations-comply-gdpr , June 2024. License: CC BY 4.0

    2024

  6. [14]

    Collins, Matthew Barker, et al

    Katherine M. Collins, Matthew Barker, et al. Human uncertainty in concept-based AI systems. In Proceedings of the Sixth AAAI/ACM Conference on Artificial Intelligence, Ethics and Society (AIES 2023), Montréal, QC, Canada, 2023

    2023

  7. [15]

    Ethics guidelines for trustworthy ai, Report / Study | Publication 08 April 2019

    European Commision. Ethics guidelines for trustworthy ai, Report / Study | Publication 08 April 2019

    2019

  8. [16]

    Explicit deepfake images of taylor swift elude safeguards and swamp social media, January 26, 2024

    Kate Conger and John Yoon. Explicit deepfake images of taylor swift elude safeguards and swamp social media, January 26, 2024

    2024

  9. [17]

    Towards guaranteed safe AI: A framework for ensuring robust and reliable AI systems

    David Dalrymple, Joar Skalse, Yoshua Bengio, Stuart Russell, Max Tegmark, Sanjit Seshia, Steve Omohundro, Christian Szegedy, Ben Goldhaber, Nora Ammann, Alessandro Abate, Joe Halpern, Clark Barrett, Ding Zhao, Tan Zhi-Xuan, Jeannette Wing, and Joshua Tenenbaum. Towards guarant...

    2024 arXiv

  10. [18]

    Scalable watermarking for identifying large language model outputs

    Sumanth Dathathri, Abigail See, Sumedh Ghaisas, Po-Sen Huang, Rob McAdam, Johannes Welbl, Vandana Bachani, Alex Kaskasoli, Robert Stanforth, Tatiana Matejovicova, Jamie Hayes, Nidhi Vyas, Majd Al Merey, Jonah Brown-Cohen, Rudy Bunel, Borja Balle, Taylan Cemgil, Zahra Ahmed, Ki...

    2024

  11. [19]

    Reimagining self-adaptation in the age of large language models, 2024

    Raghav Donakanti, Prakhar Jain, Shubham Kulkarni, and Karthik Vaidhyanathan. Reimagining self-adaptation in the age of large language models, 2024

    2024

  12. [20]

    Connecting the dots in trustworthy artificial intelligence: From AI principles, ethics, and key requirements to responsible AI systems and regulation

    Natalia Díaz-Rodríguez, Javier Del Ser, Mark Coeckelbergh, Marcos López de Prado, Enrique Herrera-Viedma, and Francisco Herrera. Connecting the dots in trustworthy artificial intelligence: From AI principles, ethics, and key requirements to responsible AI systems and regulatio...

    2023

  13. [21]

    Charles E. Ebeling. Reliability and Maintainability Engineering. McGraw-Hill, New York, 1st edition, 1997

    1997

  14. [22]

    Mica R. Endsley. Toward a theory of situation awareness in dynamic systems. Human Factors, 37(1):32–64, 1995. 30

    1995

  15. [23]

    Economic impact of resilience investments in critical infrastructure systems

    Bilal Erkan and Sait Yildirim. Economic impact of resilience investments in critical infrastructure systems. Journal of Risk and Reliability, 234(1):45–56, 2020

    2020

  16. [24]

    The ai risk repository: A comprehensive meta-review, database, and taxonomy of risks from artificial intelligence, 2023

    Peter Slattery et al. The ai risk repository: A comprehensive meta-review, database, and taxonomy of risks from artificial intelligence, 2023

    2023

  17. [25]

    Compl-ai framework: A technical interpretation and llm benchmarking suite for the eu artificial intelligence act, 2023

    Philipp Guldimann et al. Compl-ai framework: A technical interpretation and llm benchmarking suite for the eu artificial intelligence act, 2023

    2023

  18. [26]

    Designing resilient and sustainable systems

    Joseph Fiksel. Designing resilient and sustainable systems. Environmental Science & Technology, 27:5330–5339, 2003

    2003

  19. [27]

    Steven W. Gilbert. Disaster resilience: A guide to the literature. NIST Special Publication 1117, 2010

    2010

  20. [28]

    K. M. Groth and A Mosleh. A data-informed pif hierarchy for model-based human reliability analysis. Reliability Engineering & System Safety, 108:154–174, 2012

    2012

  21. [29]

    On the democratic role of news recommenders

    Natali Helberger. On the democratic role of news recommenders. Digital Journalism, 7(8):993–1012, 2019

    2019

  22. [30]

    A. Heng, S. Zhang, A. C. C. Tan, and J. Mathew. Rotating machinery prognostics: State of the art, challenges and opportunities. Mechanical Systems and Signal Processing, 23(3):724–739, 2009

    2009

  23. [31]

    Woods, and Nancy Leveson

    Erik Hollnagel, David D. Woods, and Nancy Leveson. Resilience Engineering: Concepts and Precepts. Ashgate, 2006

    2006

  24. [32]

    Woods, and Nancy Levensen

    Erik Hollnagel, David E. Woods, and Nancy Levensen. Resilience Engineering: Concepts and Precepts. Ashgate Publishing, 2006

    2006

  25. [33]

    Freeman, and Xinwei Deng

    Yili Hong, Jiayi Lian, Li Xu, Jie Min, Yueyao Wang, Laura J. Freeman, and Xinwei Deng. Statistical perspectives on reliability of artificial intelligence systems. Quality Engineering, 35(1):56–78, 2023

    2023

  26. [34]

    King, and William Q

    Yili Hong, Jie Min, Caleb B. King, and William Q. Meeker. Reliability analysis of artificial intelligence systems using recurrent events data from autonomous vehicles, 2021

    2021

  27. [35]

    Accelerated life testing (ALT) design based on computational reliability analysis

    Zhen Hu and Sankaran Mahadevan. Accelerated life testing (ALT) design based on computational reliability analysis. Quality and Reliability Engineering International, 2017. Accessed: 2024-09-01

    2017

  28. [36]

    Advancing modern code review effectiveness through human error mechanisms

    Fuqun Huang and Henrique Madeira. Advancing modern code review effectiveness through human error mechanisms. Journal of Systems and Software, 214:112060, 2024

    2024

  29. [37]

    Artificial intelligence for industry 4.0: Systematic review of applications, challenges, and opportunities

    Zohaib Jan, Farhad Ahamed, Wolfgang Mayer, Niki Patel, Georg Grossmann, Markus Stumptner, and Ana Kuusk. Artificial intelligence for industry 4.0: Systematic review of applications, challenges, and opportunities. Expert Systems with Applications, page 119456, 2022

    2022

  30. [38]

    Artificial intelligence for industry 4.0: Systematic review of applications, challenges, and opportunities

    Zohaib Jan, Farhad Ahamed, Wolfgang Mayer, Niki Patel, Georg Grossmann, Markus Stumptner, and Ana Kuusk. Artificial intelligence for industry 4.0: Systematic review of applications, challenges, and opportunities. Expert Systems with Applications, 216:119456, 2023

    2023

  31. [39]

    Failure modes in machine learning, 2024

    Ram Shankar Siva Kumar, David O’Brien, Kendra Albert, Salome Viljoen, and Jeffrey Snover. Failure modes in machine learning, 2024. Microsoft Report, July 2024

    2024

  32. [40]

    AI safety on whose terms? Science, 381(6654):138, 2023

    Seth Lazar and Alondra Nelson. AI safety on whose terms? Science, 381(6654):138, 2023

    2023

  33. [41]

    Should artificial intelligence have lower acceptable error rates than humans? Insights into Imaging, 14(1):79, 2023

    Anders Lenskjold, Janus Uhd Nybing, Charlotte Trampedach, Astrid Galsgaard, Mathias Willadsen Brejnebøl, Henriette Raaschou, Martin Høyer Rose, and Mikael Boesen. Should artificial intelligence have lower acceptable error rates than humans? Insights into Imaging, 14(1):79, 2023

    2023

  34. [42]

    Chatgpt-like large-scale foundation models for prognostics and health management: A survey and roadmaps, 2023

    Yan-Fu Li, Huan Wang, and Muxia Sun. Chatgpt-like large-scale foundation models for prognostics and health management: A survey and roadmaps, 2023

    2023

  35. [43]

    Artificial intelligence index report 2024, 2024

    Nestor Maslej, Loredana Fattorini, Raymond Perrault, Vanessa Parli, Anka Reuel, Erik Brynjolfsson, John Etchemendy, Katrina Ligett, Terah Lyons, James Manyika, Juan Carlos Niebles, Yoav Shoham, Russell Wald, and Jack Clark. Artificial intelligence index report 2024, 2024

    2024

  36. [44]

    Software reliability growth models predict autonomous vehicle disengagement events, 2018

    Robert Merkel. Software reliability growth models predict autonomous vehicle disengagement events, 2018

    2018

  37. [45]

    The ai incident database as an educational tool to raise awareness of ai harms: A classroom exploration of efficacy, limitations, & future improvements, 2023

    Hoda Heidari Michael Feffer, Nikolas Martelaro. The ai incident database as an educational tool to raise awareness of ai harms: A classroom exploration of efficacy, limitations, & future improvements, 2023

    2023

  38. [46]

    Robert Miller and A. D. Swain. Human error and human reliability. In Gavriel Salvendy, editor, Handbook of Human Factors. Wiley, 1987

    1987

  39. [47]

    A family of software reliability models with bathtub-shaped fault detection rate

    Maskura Nafreen and Lance Fiondella. A family of software reliability models with bathtub-shaped fault detection rate. International Journal of Reliability, Quality and Safety Engineering, 28(05):2150034, 2021. 31

    2021

  40. [48]

    Nasa human reliability analysis handbook with an emphasis on nuclear power plant applications

    NASA. Nasa human reliability analysis handbook with an emphasis on nuclear power plant applications. Technical memorandum, NASA, 2000

    2000

  41. [49]

    Collision between vehicle controlled by developmental automated driving system and pedestrian

    National Transportation Safety Board. Collision between vehicle controlled by developmental automated driving system and pedestrian. Accident Report NTSB/HAR-19/03, National Transportation Safety Board, 2019

    2019

  42. [50]

    Talking to bots: Symbiotic agency and the case of tay

    Gina Neff and Peter Nagy. Talking to bots: Symbiotic agency and the case of tay. International Journal of Communication, 10:4915–4931, 2016

    2016

  43. [51]

    The Filter Bubble: What the Internet Is Hiding from You

    Eli Pariser. The Filter Bubble: What the Internet Is Hiding from You. Penguin Press, 2011

    2011

  44. [52]

    A taxonomic system for failure cause analysis of open source AI incidents

    Nikiforos Pittaras and Sean McGregor. A taxonomic system for failure cause analysis of open source AI incidents. https://doi.org/10.48550/arXiv.2211.07280, 2023

    2023 doi arXiv

  45. [53]

    How a portfolio approach to AI helps your ROI

    PwC. How a portfolio approach to AI helps your ROI. https://www.pwc.com/us/en/tech-effect/ ai-analytics/how-ai-portfolio-helps-roi.html , 2021. Accessed: 9/1/2023

    2021

  46. [54]

    Solving AI’s ROI problem

    PwC. Solving AI’s ROI problem. it’s not that easy. Available online at: https://www.pwc.com/us/en/ tech-effect/ai-analytics/artificial-intelligence-roi.html , 2023

    2023

  47. [55]

    Ramasso and A

    E. Ramasso and A. Saxena. Performance benchmarking and analysis of prognostic methods for cmapss datasets. International Journal of Prognostics and Health Management, 5(2), 2014

    2014

  48. [56]

    Anand S. Rao. Foundations of operationalizing AI: Value scoping, 2023. Lecture notes available upon request

    2023

  49. [57]

    Pejtersen, and L

    Jens Rasmussen, Annelise M. Pejtersen, and L. P. Goodstein. Cognitive Systems Engineering. Wiley, 1994

    1994

  50. [58]

    Predicting software field reliability

    Pete Rotella, Sunita Chulani, and Devesh Goyal. Predicting software field reliability. In Proceedings of the Second International Workshop on Software Engineering Research and Industrial Practice, page 62–65. IEEE Press, 2015

    2015

  51. [59]

    A survey on cyber- resilience approaches for cyber-physical systems

    Mariana Segovia-Ferreira, Jose Rubio-Hernan, Ana Cavalli, and Joaquin Garcia-Alfaro. A survey on cyber- resilience approaches for cyber-physical systems. ACM Comput. Surv., 56(8), apr 2024

    2024

  52. [60]

    X.-S. Si, W. Wang, C.-H. Hu, and D.-H. Zhou. Remaining useful life estimation–a review on the statistical data-driven approaches. European Journal of Operational Research, 213(1):1–14, 2011

    2011

  53. [61]

    Cyber resilience in complex systems: An economic perspective

    John Smith and Anisha Patel. Cyber resilience in complex systems: An economic perspective. International Journal of Information Management, 58:102279, 2021

    2021

  54. [62]

    A three-parameter fault-detection software reliability model with the uncertainty of operating environments

    Kwang Yoon Song, In Hong Chang, and Hoang Pham. A three-parameter fault-detection software reliability model with the uncertainty of operating environments. Journal of Systems Science and Systems Engineering , 26(1):121–132, feb 2017

    2017

  55. [63]

    Stanton, Paul M

    Neville A. Stanton, Paul M. Salmon, Guy H. Walker, Chris Baber, and Daniel P. Jenkins.Human Factors Methods: A Practical Guide for Engineering and Design. CRC Press, 2017

    2017

  56. [64]

    Condition monitoring using machine learning: A review of theory, applications, and recent advances

    Onur Surucu, Stephen Andrew Gadsden, and John Yawney. Condition monitoring using machine learning: A review of theory, applications, and recent advances. Expert Systems with Applications, 221:119738, 2023

    2023

  57. [65]

    A. D. Swain and H. E. Guttmann. Handbook of human reliability analysis with emphasis on nuclear power plant applications. NUREG/CR Report NUREG/CR-1278, U.S. Nuclear Regulatory Commission, 1983

    1983

  58. [66]

    Artificial intelligence for safety and reliability: A descriptive, bibliometric and interpretative review on machine learning

    Nicola Tamascelli, Alessandro Campari, Tarannom Parhizkar, and Nicola Paltrinieri. Artificial intelligence for safety and reliability: A descriptive, bibliometric and interpretative review on machine learning. Journal of Loss Prevention in the Process Industries, 90:105343, 2024

    2024

  59. [67]

    AI-generated deepfakes flood social media amid political concerns, 2024

    Financial Times. AI-generated deepfakes flood social media amid political concerns, 2024

    2024

  60. [68]

    Fault diagnosis of the autonomous driving perception system based on information fusion

    Pengyu Li Wenkui Hou, Wanyu Li. Fault diagnosis of the autonomous driving perception system based on information fusion. Sensors, 32(11):446–467, 2023

    2023

  61. [69]

    R. M. Wideman. Project management consultant, composite additions from various sources 1998-2017, 2017

    1998

  62. [70]

    A. Wood. Software reliability growth models, 1996

    1996

  63. [71]

    Continual learning for large language models: A survey, 2024

    Tongtong Wu, Linhao Luo, Yuan-Fang Li, Shirui Pan, Thuy-Trang Vu, and Gholamreza Haffari. Continual learning for large language models: A survey, 2024

    2024

  64. [72]

    Ng, Curtis P

    Feiyang Yu, Mark Endo, Rayan Krishnan, Ian Pan, Andy Tsai, Eduardo Pontes Reis, Eduardo Kaiser Uru- rahy Nunes Fonseca, Henrique Min Ho Lee, Zahra Shakeri Hossein Abad, Andrew Y . Ng, Curtis P. Langlotz, Vasantha Kumar Venugopal, and Pranav Rajpurkar. Evaluating progress in au...

    2023

  65. [73]

    The Monte Carlo Simulation Method for System Reliability and Risk Analysis

    Enrico Zio. The Monte Carlo Simulation Method for System Reliability and Risk Analysis. Number 978-1-4471- 4588-2 in Springer Series in Reliability Engineering. Springer, September 2013. 32

    2013

  66. [74]

    bathtub curve

    Éder S. Brito, Vera L.D. Tomazella, and Paulo H. Ferreira. Statistical modeling and reliability analysis of multiple repairable systems with dependent failure times under perfect repair. Reliability Engineering & System Safety, 222:108375, 2022. 33 Supporting Information A Bac...

    2022

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.