Assessing the Safety and Reliability of Autonomous Vehicles from Road Testing

David Flynn; Kizito Salako; Lorenzo Strigini; Valentin Robu; Xingyu Zhao

REVIEW 2 major objections 6 minor 48 references

Assessing the Safety and Reliability of Autonomous Vehicles from Road Testing

T0 review · 2 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read With only two quantile constraints on an AV fatality rate, a worst-case Bayesian prior yields conservative posterior confidence and can cut required road-test mileage by about three quarters when prior confidence is strong.

desk verdict The new CBI theorem for non-zero failure counts is sound and worth knowing, but the SRGM half of the paper rests on an unreported synthetic preprocessing step that makes its empirical results hard to trust as presented. read the letter →

arxiv 1908.06540 v1 pith:GBHMRKC5 submitted 2019-08-19 cs.AI cs.CYcs.SE

Xingyu Zhao , Valentin Robu , David Flynn , Kizito Salako , Lorenzo Strigini This is my paper · ORCID

classification cs.AIcs.CYcs.SE

keywords autonomousvehiclesconservativeBayesianinferencereliabilityassessmentpriorknowledgesoftwaregrowthmodelsdisengagementdatarecalibrationroadtesting

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper targets a practical impasse: demonstrating the very low fatality rates expected of autonomous vehicles from road testing alone would require hundreds of millions of miles, while most published analyses ignore evidence available before testing. It proposes a variant of Conservative Bayesian Inference in which the assessor needs only two numbers about the unknown fatality rate $X$: a prior confidence $\theta$ that $X$ is below the engineering goal $\epsilon$, and certainty that $X$ is above a physical lower bound $pl$. Theorem 1 shows that, among all priors satisfying those constraints, the one minimizing posterior confidence in any claimed bound $p$ is a two-point distribution, with the worst-case confidence given explicitly by Eq. (3) for any observed failure count $k$. This generalizes CBI from failure-free testing to rare failures, and the paper uses it to compare required test miles with classical and standard Bayesian approaches. For disengagements, it fits several software reliability growth models to 51 months of public road-test records, checks their forecast accuracy, and shows recalibration brings their median-miles-to-next-disengagement forecasts to around 7,000--8,000 miles.

What carries the argument

The load-bearing object is the conservative two-point prior: a prior distribution placing probability $\theta$ at a lower point $x_1$ between the physical lower bound $pl$ and the engineering goal $\epsilon$, and probability $1-\theta$ at an upper point $x_3$ above $\epsilon$. Theorem 1 proves this is the feasible prior that minimizes the posterior probability of the safety bound $p$. The argument works in three stages: first any feasible prior is replaced by an equivalent three-point prior; then the middle mass is shown to increase the posterior, so it is set to zero; finally $x_1$ and $x_3$ are chosen by monotonicity of the likelihood factor $x^k(1-x)^{n-k}$. That factor carries the whole computation, turning an infinite-dimensional optimization over priors into a two-point placement from which mileage requirements follow directly from Eq. (3).

What would settle it

Take a concrete instance of the theorem, say $\epsilon=1.09\times10^{-10}$, $pl=10^{-15}$, $p=1.09\times10^{-8}$, $\theta=0.9$, $k=0$, $n=69\times10^6$, and numerically minimize the posterior confidence in Eq. (1) over a dense family of feasible priors; any feasible prior giving posterior confidence below the value from Eq. (3) would refute the claimed conservatism.

Watch

Extended reading notes

Core claim

On its own terms, the discovery is that conservatism and prior knowledge are compatible: a Bayesian assessor does not need a full prior distribution to obtain a sound posterior bound. For a Bernoulli per-mile failure process with unknown rate $X$, given constraints $\Pr(X\le\epsilon)=\theta$ and $\Pr(X>pl)=1$, the worst-case posterior confidence $\Pr(X\le p \mid k \text{ failures in } n \text{ miles})$ is attained by a two-point prior with mass $\theta$ at $x_1\in[pl,\epsilon]$ and mass $1-\theta$ at $x_3>\epsilon$. The minimized value is the rational expression in Eq. (3), which is zero for $p\le\epsilon$ and otherwise depends on $x_1^k(1-x_1)^{n-k}\theta$ relative to $x_3^k(1-x_3)^{n-k}(1-\theta)$. The proof collapses any feasible prior to a three-point prior, shows the middle mass can be removed, and then places the two remaining points by monotonicity of $x^k(1-x)^{n-k}$, so the required number of test miles can be solved directly. A direct corollary is that under these partial prior constraints, no amount of failure-free testing supports a safety bound better than the engineering goal $\epsilon$.

Load-bearing premise

The load-bearing premise is that an assessor can state a trustworthy prior confidence $\theta$ in the engineering goal and a physical lower bound $pl$, and that the per-mile failure probability is constant over the assessment period; if those prior numbers are not defensible, the mileage reductions the method offers are not defensible either.

Editorial extensions

If this is right

If an assessor can justify a strong prior confidence $\theta=0.9$ in the engineering goal, CBI supports a 95% claim on a human-comparable fatality bound with about 69 million fatality-free miles, versus 275 million under the classical confidence approach; a weak prior $\theta=0.1$ raises the requirement to about 476 million miles.
Under the stated partial prior constraints, no amount of failure-free road testing can support any fatality-rate claim better than the engineering goal $\epsilon$; CBI makes this limit explicit rather than hidden in a prior distribution.
When observed failures are included, CBI demands substantially more miles than classical or uniform-prior approaches (roughly 79 billion versus 5.0 billion miles in the 43-fatality scenario), reflecting its avoidance of optimistic prior assumptions.
After one observed fatality, the additional fatality-free miles needed to restore a given claim has a ceiling of $1/\epsilon$ once the initial test mileage is large, and this ceiling is independent of the confidence levels $c$ and $\theta$.
Recalibrated software reliability growth models applied to 51 months of disengagement records converge near a current median-miles-to-disengagement estimate of 7,000--8,000 miles, making them a practical test-planning aid rather than a tool for demonstrating ultra-high safety.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A consequence the paper leaves implicit is that the same worst-case-prior construction applies to any safety-critical machine-learning system with a verifiable non-ML backstop: whenever an assessor can state $\epsilon$, $\theta$ and $pl$, Eq. (3) determines the operational evidence required, so the paper's mileage results can be read as generic evidence requirements.
A testable extension is to derive $\theta$ from simulation, formal verification, or component testing results rather than eliciting it as a judgement; the paper lists these as possible sources but does not develop a calibration procedure that turns such evidence into the prior confidence value.
The same forecast-recalibration pipeline could be applied to other fleets' monthly disengagement reports; because those reports are monthly aggregates, exact event dates would make third-party reliability-growth forecasts fully reproducible, whereas reconstructed event times leave the forecasts dependent on the reconstruction procedure.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

The thing to know about this paper: the theoretical half is genuinely good, and the empirical half is less solid than it looks. The CBI result generalizes existing conservative Bayesian inference from failure-free testing to cases with k > 0 failures, and the proof in Appendix A is constructive and checks out. The numerical illustrations (Figures 2-4, Table I) reproduce from the stated equations and parameters, and the comparison with the RAND study is fair. That part deserves credit.

The soft spot is the SRGM contribution. The authors take Waymo's monthly disengagement counts and miles, then generate exact inter-failure mileages by randomly placing points inside each month under a Poisson assumption. They mention repeating this to check sensitivity, but they do not report the sensitivity results or a random seed. That means the u-plots, PLR-plots, recalibration curves, and the headline 7,000-8,000 mile MMTD estimate all depend on an arbitrary synthetic sequence that no one can reproduce from the manuscript. No code or data artifacts are provided either. This is not a minor omission; it is the load-bearing step for the empirical half. The authors are honest that SRGMs are not suitable for safety claims, but they do claim SRGMs are useful as planning aids, and that claim is currently supported only by a non-reproducible preprocessing step.

To be fair, the CBI theorem is exactly what the paper says it is: a self-contained extension of prior work, and the self-citations to earlier CBI papers are appropriate since this is a direct generalization. The SRGM methodology is standard, and the paper makes no pretense of inventing it. The weakness is not in the theory but in the execution of the empirical demonstration.

I would send this to a serious referee. The theoretical result is worth publishing, and the SRGM application, if the authors can either provide the synthetic sequences or analyze the sensitivity transparently, could be made reproducible. The paper is written clearly and the authors know their limitations, which is more than many in this space. If I were refereeing, I would ask for the sensitivity analysis to be shown or the random seed released, and for confidence intervals on the MMTD predictions. With that, the paper would be a solid contribution to the AV safety literature.

For a reading group, it is a reasonable choice if the group cares about conservative Bayesian methods; the SRGM part is more of a cautionary tale about preprocessing.

Referee Report

2 major / 6 minor

Summary. This paper addresses two related problems in assessing autonomous vehicle safety from road testing. First, it develops a Conservative Bayesian Inference (CBI) framework for claims about a constant per-mile failure probability (pfm/pcm) when testing produces a small number of failures. Theorem 1 identifies the prior distribution, subject to partial prior knowledge Pr(X≤ε)=θ and Pr(X>pl)=1, that minimizes the posterior confidence in any bound p; the result generalizes existing CBI from failure-free testing to arbitrary failure counts. The authors compare the required number of test miles against the classical RAND approach and standard Bayesian priors, and derive asymptotic properties for the number of extra miles needed to compensate for a fatality (Q3). Second, the paper applies Software Reliability Growth Models (SRGMs) to Waymo's public disengagement data over 51 months, with u-plot and PLR-plot accuracy assessment and recalibration, concluding that SRGMs can be a useful planning aid. The abstract and introduction frame this as a contribution to safety and reliability assessment of autonomous vehicles.

Significance. The CBI theorem is a rigorous and useful extension: its proof in Appendix A is constructive and self-contained, and the numerical comparisons in Figures 2–4 and Table I follow from the stated formulas. If the SRGM empirical analysis were fully reproducible, the paper would provide a valuable demonstration of how forecast-accuracy assessment and recalibration can make SRGMs usable for planning AV disengagement trends. The paper explicitly avoids overclaiming SRGMs for safety certification, which is appropriate. The main significance is the new conservative Bayesian method for combining partial prior knowledge with sparse failure evidence; the SRGM application is a secondary, methodological contribution.

major comments (2)

[Section IV-A, footnote 8] The preprocessing of Waymo's monthly data into a sequence of inter-failure mileages is described only in a footnote as 'generating random points in a Poisson Process for each month, repeating to check sensitivity of the results to this manipulation,' with no details of the procedure, the number of replications, the random seed, or the results of the sensitivity checks. Consequently, the MMTD predictions, u-plots, PLR-plots, and the headline MMTD estimate of 7,000–8,000 miles are not independently reproducible and may depend on the particular synthetic sequence generated. This is a load-bearing deficiency because the paper's second contribution is an empirical demonstration that SRGMs with recalibration are useful planning aids. The authors should report the full preprocessing protocol, the sensitivity results (e.g., the range of MMTD predictions across replications), and make the code and data available, or explicitly reframe the results as conditional on a specific synthetic realization.
[Section IV-A, Figure 5C] The statement 'the best estimate of current MMTD is thus about 7-8000 miles' is a point estimate with no uncertainty quantification. Given that the SRGMs in Figure 5A produce widely different MMTD predictions, and given the stochastic preprocessing, the paper should provide confidence intervals or a sensitivity range for this estimate, for example by showing the distribution of recalibrated MMTD predictions over the random replications. Without this, the claim that SRGMs can provide practical planning forecasts is not quantitatively supported.

minor comments (6)

[Section IV-A] The abbreviations GO, MO, LV, and Li are used in the text and figure without a complete mapping in the main body; consider placing the model names in the caption of Figure 5 or in a separate table.
[Appendix B] Equation (10) appears to involve a floor or ceiling function, but this notation is not defined in the main text; please clarify the operation used.
[Section IV-A] The paper refers to the PETERS toolset without providing a citation or a description of its provenance; a reference or a brief explanation would help a reader locate the tool.
[Section III-B, Q2] The phrase 'we thus postulate an observed number of fatalities' should explicitly state that the k=43 value is a hypothetical expectation used for comparison, not an actual observation, to avoid confusion.
[Throughout] The text consistently uses 'A Vs' with a space; this should be standardized to 'AVs' for stylistic consistency.
[Section IV-A, Figure 5] The sentence 'In Fig. 5A,C,E,F, the 528 failures...' is confusing because the panels have different roles; consider rephrasing to describe the content of each panel explicitly.

Circularity Check

0 steps flagged · score 2.0 of 10

No circularity in the central CBI derivation; minor self-citations and an unreported preprocessing detail explain the low non-circularity score.

full rationale

The CBI section is self-contained: Theorem 1 is proved in Appendix A by an explicit three-stage minimization over prior distributions, and Eq. (3) is a derived worst-case posterior, not an input assumption. The numerical CBI results are computed from Eq. (3)/Eq. (10) with stated parameters, so they are not fitted outputs. The SRGM half uses sequential forecasting: each MMTD prediction is made from previous inter-failure miles only and is then scored against the next observed disengagement via u-plots and PLR-plots; recalibration is a standard correction based on past prediction errors, so forecasts are not equal to fitted inputs. The authors cite their own earlier CBI papers ([23]-[26], [38]) for background and for the failure-free special case, but the generalization to k>0 is proved here, so the self-citations are not load-bearing. The one genuine concern is footnote 8: Waymo's monthly disengagement counts and mileages are converted to an inter-failure sequence by "generating random points in a Poisson Process for each month, repeating to check sensitivity of the results to this manipulation," but the sensitivity results and random seed are not reported. This makes the SRGM/MMTD numbers hard to reproduce, but it is a data preprocessing assumption, not a claim that reduces by construction to its own inputs. Therefore no circular step is established.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The CBI theorem is a mathematical result conditional on the Bernoulli model and the partial prior constraints; these are domain assumptions, not fitted. The SRGM analysis additionally depends on an ad hoc synthetic data generation step.

free parameters (4)

epsilon (engineering goal) = 1.09e-10 (illustrative)
Target pfm two orders of magnitude safer than human drivers; chosen for the numerical example in Section III-B, not fitted to data.
pl (lower bound on pfm) = 1e-15
Assumed physical lower bound due to hardware failures; used in the CBI constraints and numerical examples.
theta (prior confidence) = 0.1 and 0.9
Prior confidence that the engineering goal is met; chosen to illustrate weak and strong priors in Section III-B.
SRGM model parameters = Not reported
Parameters of the 8 SRGMs fitted to Waymo disengagement data in Section IV-A; values not given in the paper, but the forecasts depend on them.

assumptions (4)

domain assumption Failures per mile follow a Bernoulli process with constant probability X
Assumed in Section II for fatalities and crashes; justifies the likelihood x^k(1-x)^(n-k) in Theorem 1.
domain assumption Partial prior knowledge Pr(X<=epsilon)=theta and Pr(X>pl)=1
Core input to CBI (Section III-A); if not true, the conservative bound is not valid.
ad hoc to paper Monthly disengagement counts can be converted to exact inter-failure miles by generating random points from a Poisson process within each month
Section IV-A; the raw Waymo data are monthly aggregates, and PETERS requires inter-failure times. This assumption is not validated and the sensitivity check is not reported.
standard math SRGM assumptions (e.g., each fault contributes to failure intensity, fixes remove faults)
Standard SRGM assumptions inherited from [27], [43]; the paper deliberately does not commit to a single model.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Assessing the Safety and Reliability of Autonomous Vehicles from Road Testing." pith.science (2026). https://pith.science/paper/GBHMRKC5

@misc{pith2026190806540,
  author       = {Pith},
  title        = {Pith review of: Assessing the Safety and Reliability of Autonomous Vehicles from Road Testing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GBHMRKC5}},
  note         = {Machine review of arXiv:1908.06540}
}

read the original abstract

There is an urgent societal need to assess whether autonomous vehicles (AVs) are safe enough. From published quantitative safety and reliability assessments of AVs, we know that, given the goal of predicting very low rates of accidents, road testing alone requires infeasible numbers of miles to be driven. However, previous analyses do not consider any knowledge prior to road testing - knowledge which could bring substantial advantages if the AV design allows strong expectations of safety before road testing. We present the advantages of a new variant of Conservative Bayesian Inference (CBI), which uses prior knowledge while avoiding optimistic biases. We then study the trend of disengagements (take-overs by human drivers) by applying Software Reliability Growth Models (SRGMs) to data from Waymo's public road testing over 51 months, in view of the practice of software updates during this testing. Our approach is to not trust any specific SRGM, but to assess forecast accuracy and then improve forecasts. We show that, coupled with accuracy assessment and recalibration techniques, SRGMs could be a valuable test planning aid.

Figures

Figures reproduced from arXiv: 1908.06540 by the authors.

**Figure 3.** Failure-free miles needed to be driven to demonstrate a less [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

**Figure 4.** Fatality-free miles needed to compensate one newly observed [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗

Figures from the paper (2 more)

**Figure 5.** Figure 5: MMTD predictions (A, improved in C), u-plots (B and [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

**Figure 6.** Figure 6: The 5 possible cases of two-point prior distributions that minimise (5). Notice the important role of where [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

48 extracted references · 47 canonical work pages

[1]

Autonomous vehicle technology: A guide for policymakers,

J. M. Anderson, K. Nidhi, K. D. Stanley, P. Sorensen, C. Samaras, and O. A. Oluwatola, “Autonomous vehicle technology: A guide for policymakers,” Rand Corporation, Tech. Rep. RR-443-2-RC, 2016

work page 2016
[2]

A survey of motion planning and control techniques for self-driving urban vehicles,

B. Paden, M. ˇC´ap, S. Z. Yong, D. Yershov, and E. Frazzoli, “A survey of motion planning and control techniques for self-driving urban vehicles,” IEEE Tran. on Intelligent Vehicles , vol. 1, no. 1, pp. 33–55, 2016

work page 2016
[3]

Preparing a nation for autonomous vehicles: Opportunities, barriers and policy recommendations,

D. J. Fagnant and K. Kockelman, “Preparing a nation for autonomous vehicles: Opportunities, barriers and policy recommendations,” Transp. Research Part A: Policy and Practice , vol. 77, pp. 167–181, 2015

work page 2015
[4]

Autonomous vehicle safety: An interdis- ciplinary challenge,

P. Koopman and M. Wagner, “Autonomous vehicle safety: An interdis- ciplinary challenge,” IEEE Intelligent Transportation Systems Magazine, vol. 9, no. 1, pp. 90–96, 2017

work page 2017
[5]

The social dilemma of autonomous vehicles,

J.-F. Bonnefon, A. Shariff, and I. Rahwan, “The social dilemma of autonomous vehicles,”Science, vol. 352, no. 6293, pp. 1573–1576, 2016

work page 2016
[6]

Planning and decision- making for autonomous vehicles,

W. Schwarting, J. Alonso-Mora, and D. Rus, “Planning and decision- making for autonomous vehicles,” Annual Review of Control, Robotics, and Autonomous Systems , vol. 1, no. 1, pp. 187–210, 2018

2018
[7]

Hands off: The future of self-driving cars,

C. Urmson, “Hands off: The future of self-driving cars,” Committee on Commerce, Science and Transportation, Washington, D.C., USA, Testimony, 2016

work page 2016
[8]

Hands off the wheel in autonomous vehicles?: A systems perspective on over a million miles of ﬁeld data,

S. S. Banerjee, S. Jha, J. Cyriac, Z. T. Kalbarczyk, and R. K. Iyer, “Hands off the wheel in autonomous vehicles?: A systems perspective on over a million miles of ﬁeld data,” in 48th IEEE/IFIP Int. Conf. on Dependable Systems and Networks , 2018, pp. 586–597

work page 2018

Show all 48 references

[9]

Driving to safety: How many miles of driving would it take to demonstrate autonomous vehicle reliability?

N. Kalra and S. Paddock, “Driving to safety: How many miles of driving would it take to demonstrate autonomous vehicle reliability?” Transp. Research Part A: Policy and Practice , vol. 94, pp. 182–193, 2016

2016
[10]

Autonomous vehicles’ disengage- ments: Trends, triggers, and regulatory limitations,

F. Favar `o, S. Eurich, and N. Nader, “Autonomous vehicles’ disengage- ments: Trends, triggers, and regulatory limitations,” Accident Analysis & Prevention, vol. 110, pp. 136 – 148, 2018

2018
[11]

Autonomous vehicles: Disen- gagements, accidents and reaction times,

V . V . Dixit, S. Chand, and D. J. Nair, “Autonomous vehicles: Disen- gagements, accidents and reaction times,” PLOS ONE, vol. 11, no. 12, pp. 1–14, 2016

2016
[12]

Analysis of autopilot disengagements occurring during autonomous vehicle testing,

C. Lv, D. Cao, Y . Zhao, D. J. Auger, M. Sullman, H. Wang, L. M. Dutka, L. Skrypchuk, and A. Mouzakitis, “Analysis of autopilot disengagements occurring during autonomous vehicle testing,” IEEE/CAA Journal of Automatica Sinica, vol. 5, no. 1, pp. 58–68, Jan. 2018

2018
[13]

Validation of ultra-high dependability for software-based systems,

B. Littlewood and L. Strigini, “Validation of ultra-high dependability for software-based systems,” Comm. of the ACM , vol. 36, pp. 69–80, 1993

1993
[14]

The infeasibility of quantifying the reliability of life-critical real-time software,

R. W. Butler and G. B. Finelli, “The infeasibility of quantifying the reliability of life-critical real-time software,” IEEE Transactions on Software Engineering, vol. 19, no. 1, pp. 3–12, Jan. 1993

1993
[15]

Reasoning about the reliability of diverse two-channel systems in which one channel is ‘possibly perfect’,

B. Littlewood and J. Rushby, “Reasoning about the reliability of diverse two-channel systems in which one channel is ‘possibly perfect’,” IEEE Tran. on Software Engineering , vol. 38, no. 5, pp. 1178–1194, 2012

2012
[16]

Waymo safety report: On the road to fully self-driving,

Waymo, “Waymo safety report: On the road to fully self-driving,” Tech. Rep., 2018. [Online]. Available: https://storage.googleapis.com/ sdc-prod/v1/safety-report/SafetyReport2018.pdf

2018
[17]

A plan to develop safe autonomous vehicles. And prove it,

A. Shashua and S. Shalev-Shwartz, “A plan to develop safe autonomous vehicles. And prove it,” Intel Newsroom , p. 8, 2017. [Online]. Available: https://newsroom.intel.com/newsroom/wp-content/ uploads/sites/11/2017/10/autonomous-vehicle-safety-strategy.pdf

2017
[18]

DeepTest: Automated testing of deep-neural-network-driven autonomous cars,

Y . Tian, K. Pei, S. Jana, and B. Ray, “DeepTest: Automated testing of deep-neural-network-driven autonomous cars,” in the 40th Int. Conf. on Software Engineering, New York, NY , USA, 2018, pp. 303–314

2018
[19]

Formal veriﬁcation of autonomous vehicle platooning,

M. Kamali, L. A. Dennis, O. McAree, M. Fisher, and S. M. Veres, “Formal veriﬁcation of autonomous vehicle platooning,” Science of Computer Programming, vol. 148, pp. 88 – 106, 2017

2017
[20]

Veriﬁable self-certifying autonomous systems,

M. Fisher, E. Collins, L. Dennis, M. Luckcuck, M. Webster, M. Jump, V . Page, C. Patchett, F. Dinmohammadi, D. Flynn, V . Robu, and X. Zhao, “Veriﬁable self-certifying autonomous systems,” in IEEE Int. Symp. on Software Reliability Engineering Workshops , 2018, pp. 341–348

2018
[21]

Safety argument considerations for public road testing of autonomous vehicles,

P. Koopman and B. Osyk, “Safety argument considerations for public road testing of autonomous vehicles,” in WCX SAE World Congress Experience. SAE International, Apr. 2019

2019
[22]

Assessing asymmetric fault-tolerant software,

P. Popov and L. Strigini, “Assessing asymmetric fault-tolerant software,” in the 21st Int. Symp. on Software Reliability Engineering . San Jose, CA, USA: IEEE Computer Society Press, 2010, pp. 41–50

2010
[23]

Toward a formalism for conservative claims about the dependability of software-based systems,

P. Bishop, R. Bloomﬁeld, B. Littlewood, A. Povyakalo, and D. Wright, “Toward a formalism for conservative claims about the dependability of software-based systems,” IEEE Transactions on Software Engineering , vol. 37, no. 5, pp. 708–717, 2011

2011
[24]

Software fault-freeness and reliability predictions,

L. Strigini and A. Povyakalo, “Software fault-freeness and reliability predictions,” in Computer Safety, Reliability, and Security , ser. LNCS, vol. 8153. Springer Berlin Heidelberg, 2013, pp. 106–117

2013
[25]

Modeling the probability of failure on demand (pfd) of a 1-out-of-2 system in which one channel is “quasi-perfect

X. Zhao, B. Littlewood, A. Povyakalo, L. Strigini, and D. Wright, “Modeling the probability of failure on demand (pfd) of a 1-out-of-2 system in which one channel is “quasi-perfect”,” Reliability Engineering & System Safety , vol. 158, pp. 230–245, 2017

2017
[26]

Conservative claims about the probability of perfection of software-based systems,

X. Zhao, B. Littlewood, A. Povyakalo, and D. Wright, “Conservative claims about the probability of perfection of software-based systems,” in 26th Int. Symp. on Software Reliability Eng. IEEE, 2015, pp. 130–140

2015
[27]

Exponential order statistic models of software reliability growth,

D. R. Miller, “Exponential order statistic models of software reliability growth,” IEEE Tran. on Software Eng., vol. 12, no. 01, pp. 12–24, 1986

1986
[28]

Techniques for prediction analysis and recalibration,

S. Brocklehurst and B. Littlewood, “Techniques for prediction analysis and recalibration,” in Handbook of Software Reliability Eng. , M. Lyu, Ed. McGraw-Hill & IEEE Computer Society Press, 1996, pp. 119–166

1996
[29]

IEC, IEC61508, Functional Safety of Electrical/ Elec- tronic/Programmable Electronic Safety Related Systems , 2010

2010
[30]

CENELEC, EN50129, Railway Applications-Communication, Signalling and processing Systems-Safety Related Electronic Systems for Sig- nalling, 2003

2003
[31]

Handbook of parameter estimation for probabilistic risk assessment,

C. Atwood, J. LaChance, H. Martz, D. Anderson, M. Englehardt, D. Whitehead, and T. Wheeler, “Handbook of parameter estimation for probabilistic risk assessment,” U.S. Nuclear Regulatory Commission, Washington, DC, Report NUREG/CR-6823, 2003

2003
[32]

Guidelines for statistical testing,

L. Strigini and B. Littlewood, “Guidelines for statistical testing,” City University London, Project Report PASCON/WO6-CCN2/TN12, 1997

1997
[33]

Bayesian nonpara- metric system reliability using sets of priors,

G. Walter, L. J. M. Aslett, and F. P. A. Coolen, “Bayesian nonpara- metric system reliability using sets of priors,” International Journal of Approximate Reasoning, vol. 80, pp. 67–88, 2017

2017
[34]

Deriving a frequentist conservative conﬁdence bound for probability of failure per demand for systems with different operational and test proﬁles,

P. Bishop and A. Povyakalo, “Deriving a frequentist conservative conﬁdence bound for probability of failure per demand for systems with different operational and test proﬁles,” Reliability Engineering & System Safety, vol. 158, pp. 246–253, 2017

2017
[35]

Imprecise probabilistic inference for software run reliability growth models

L. V . Utkin and F. P. A. Coolen, “Imprecise probabilistic inference for software run reliability growth models.” Journal of Uncertain Systems. , vol. 12, no. 4, pp. 292–308, 2018

2018
[36]

Ex- amining accident reports involving autonomous vehicles in California,

F. M. Favar `o, N. Nader, S. O. Eurich, M. Tripp, and N. Varadaraju, “Ex- amining accident reports involving autonomous vehicles in California,” PLOS ONE, vol. 12, no. 9, pp. 1–20, 2017

2017
[37]

Cinlar, Introduction to Stochastic Processes , ser

E. Cinlar, Introduction to Stochastic Processes , ser. Dover Books on Mathematics Series. Dover Publications, Incorporated, 2013

2013
[38]

Conservative claims for the probability of perfection of a software- based system using operational experience of previous similar systems,

X. Zhao, B. Littlewood, A. Povyakalo, L. Strigini, and D. Wright, “Conservative claims for the probability of perfection of a software- based system using operational experience of previous similar systems,” Reliability Engineering & System Safety , vol. 175, pp. 265 – 282, 2018

2018
[39]

How safe is safe enough for self-driving vehicles?

P. Liu, R. Yang, and Z. Xu, “How safe is safe enough for self-driving vehicles?” Risk Analysis, vol. 39, no. 2, pp. 315–325, 2019

2019
[40]

The Moral Machine experiment,

E. Awad, S. Dsouza, R. Kim, J. Schulz, J. Henrich, A. Shariff, J.-F. Bonnefon, and I. Rahwan, “The Moral Machine experiment,” Nature, vol. 563, no. 7729, pp. 59–64, 2018

2018
[41]

Could Fisher, Jeffreys and Neyman have agreed on testing?

J. O. Berger, “Could Fisher, Jeffreys and Neyman have agreed on testing?” Statistical Science, vol. 18, no. 1, pp. 1–32, 2003

2003
[42]

Evaluation of com- peting software reliability predictions,

A. A. Abdel-Ghaly, P. Y . Chan, and B. Littlewood, “Evaluation of com- peting software reliability predictions,” IEEE Transactions on Software Engineering, vol. SE-12, no. 9, pp. 950–967, 1986

1986
[43]

New ways to get accurate reliability measures,

S. Brocklehurst and B. Littlewood, “New ways to get accurate reliability measures,” IEEE Software, vol. 9, pp. 34–42, 1992

1992
[44]

Probabilistic model checking of robots deployed in extreme environments,

X. Zhao, V . Robu, D. Flynn, F. Dinmohammadi, M. Fisher, and M. Web- ster, “Probabilistic model checking of robots deployed in extreme environments,” in Proc. of the 33rd AAAI Conference on Artiﬁcial Intelligence, vol. 33, Honolulu, Hawaii, USA, 2019, pp. 8076–8084

2019
[45]

Xie, Software reliability modelling

M. Xie, Software reliability modelling. World Scientiﬁc, 1991, vol. 1

1991
[46]

A software reliability model for artiﬁcial intelligence programs,

F. B. Bastani, I.-R. Chen, and T.-W. Tsao, “A software reliability model for artiﬁcial intelligence programs,” Int. Journal of Software Engineering and Knowledge Engineering , vol. 3, no. 01, pp. 99–114, 1993

1993
[47]

The increasing risks of risk assessment: On the rise of artiﬁcial intelligence and non-determinism in safety-critical systems,

Johnson, C. W., “The increasing risks of risk assessment: On the rise of artiﬁcial intelligence and non-determinism in safety-critical systems,” in the 26th Safety-Critical Systems Symposium. York, UK.: Safety-Critical Systems Club, 2018, p. 15

2018
[48]

Recalibrating software reliability models,

S. Brocklehurst, P. Y . Chan, B. Littlewood, and J. Snell, “Recalibrating software reliability models,” IEEE Transactions on Software Engineer- ing, vol. 16, no. 4, pp. 458–470, Apr. 1990

1990

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

Autonomous vehicle technology: A guide for policymakers,

J. M. Anderson, K. Nidhi, K. D. Stanley, P. Sorensen, C. Samaras, and O. A. Oluwatola, “Autonomous vehicle technology: A guide for policymakers,” Rand Corporation, Tech. Rep. RR-443-2-RC, 2016

work page 2016

[2] [2]

A survey of motion planning and control techniques for self-driving urban vehicles,

B. Paden, M. ˇC´ap, S. Z. Yong, D. Yershov, and E. Frazzoli, “A survey of motion planning and control techniques for self-driving urban vehicles,” IEEE Tran. on Intelligent Vehicles , vol. 1, no. 1, pp. 33–55, 2016

work page 2016

[3] [3]

Preparing a nation for autonomous vehicles: Opportunities, barriers and policy recommendations,

D. J. Fagnant and K. Kockelman, “Preparing a nation for autonomous vehicles: Opportunities, barriers and policy recommendations,” Transp. Research Part A: Policy and Practice , vol. 77, pp. 167–181, 2015

work page 2015

[4] [4]

Autonomous vehicle safety: An interdis- ciplinary challenge,

P. Koopman and M. Wagner, “Autonomous vehicle safety: An interdis- ciplinary challenge,” IEEE Intelligent Transportation Systems Magazine, vol. 9, no. 1, pp. 90–96, 2017

work page 2017

[5] [5]

The social dilemma of autonomous vehicles,

J.-F. Bonnefon, A. Shariff, and I. Rahwan, “The social dilemma of autonomous vehicles,”Science, vol. 352, no. 6293, pp. 1573–1576, 2016

work page 2016

[6] [6]

Planning and decision- making for autonomous vehicles,

W. Schwarting, J. Alonso-Mora, and D. Rus, “Planning and decision- making for autonomous vehicles,” Annual Review of Control, Robotics, and Autonomous Systems , vol. 1, no. 1, pp. 187–210, 2018

2018

[7] [7]

Hands off: The future of self-driving cars,

C. Urmson, “Hands off: The future of self-driving cars,” Committee on Commerce, Science and Transportation, Washington, D.C., USA, Testimony, 2016

work page 2016

[8] [8]

Hands off the wheel in autonomous vehicles?: A systems perspective on over a million miles of ﬁeld data,

S. S. Banerjee, S. Jha, J. Cyriac, Z. T. Kalbarczyk, and R. K. Iyer, “Hands off the wheel in autonomous vehicles?: A systems perspective on over a million miles of ﬁeld data,” in 48th IEEE/IFIP Int. Conf. on Dependable Systems and Networks , 2018, pp. 586–597

work page 2018