pith. sign in

arxiv: 2604.05662 · v1 · submitted 2026-04-07 · 💻 cs.SE · cs.CY

Understanding: reframing automation and assurance

Robin Bloomfield This is my paper

Pith reviewed 2026-05-10 19:28 UTC · model grok-4.3

classification 💻 cs.SE cs.CY
keywords understandingassurance casesautomationsafety engineeringdecision makingsocio-technical systemsargumentationepistemology
0
0 comments X

The pith

Understanding must be made an explicit, assessable component of decisions about critical systems.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper contends that safety and assurance cases for socio-technical systems risk becoming detached from the genuine human comprehension needed for responsible engineering and governance. Pressures including faster development tempos, reduced scrutiny, software complexity, and AI-generated artefacts can yield documents that appear coherent while failing to support real insight into system behavior, evidence, assumptions, risks, and residual uncertainty. Drawing on philosophical accounts of understanding, the authors propose treating it as a required, challengeable element in decision making. They outline an engineering path using Assurance 2.0 with structured argumentation to create two linked artefacts that operationalize this requirement. This reframing matters because decisions rest on what people actually grasp rather than on the surface quality of produced artefacts.

Core claim

We argue that understanding should become an explicit, assessable, and defensible component of decision making: what developers, assessors, and decision makers grasp about system behavior, evidence, assumptions, risks, and residual uncertainty. Drawing on Catherine Elgin's epistemology of understanding, we outline a conceptual foundation and then use Assurance 2.0 as an engineering route to operationalize using structured argumentation, evidence, confidence, defeaters, and theory based automation. This leads to two linked artefacts: an Understanding Basis, which justifies why available understanding is sufficient for a decision, and a Personal Understanding Statement, through which each of a

What carries the argument

Assurance 2.0, which applies structured argumentation, evidence, confidence levels, defeaters, and theory-based automation to produce an Understanding Basis that justifies sufficiency of knowledge for a decision and Personal Understanding Statements that make individual grasp explicit and challengeable.

If this is right

  • Assurance cases will include an Understanding Basis that explicitly justifies why the available understanding suffices for the decision.
  • Participants will produce Personal Understanding Statements that declare their grasp of behaviors, evidence, assumptions, risks, and uncertainties in challengeable form.
  • Automation used to generate artefacts will be evaluated both for its contribution to artefact quality and for its impact on human comprehension.
  • Evaluation of the approach will examine both efficacy of the artefacts and their epistemic effects on decision quality.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The approach could extend to regulatory domains such as financial oversight or environmental permitting where AI tools increasingly generate supporting documents.
  • Testable extensions include trials that measure whether readers of the new statements can more accurately predict system failure modes or residual uncertainties than readers of standard cases.
  • Related problems in explainable AI might adopt similar requirements for defensible personal statements of grasp rather than relying solely on generated explanations.

Load-bearing premise

That pressures for increased tempo, reduced scrutiny, software complexity, and AI-generated artefacts produce outputs that appear coherent without supporting genuine human comprehension, and that adding structured artefacts will address the problem without introducing new detachment.

What would settle it

A controlled study in which teams using the Understanding Basis and Personal Understanding Statements show no measurable improvement in their ability to identify, articulate, and address system risks and uncertainties compared with teams using conventional assurance cases.

read the original abstract

Safety and assurance cases risk becoming detached from the understanding needed for responsible engineering and governance decisions. More broadly, the production and evaluation of critical socio-technical systems increasingly face an understanding challenge: pressures for increased tempo, reduced scrutiny, software complexity, and growing use of AI generated artefacts may produce outputs that appear coherent without supporting genuine human comprehension. We argue that understanding should become an explicit, assessable, and defensible component of decision making: what developers, assessors, and decision makers grasp about system behavior, evidence, assumptions, risks, and residual uncertainty. Drawing on Catherine Elgin's epistemology of understanding, we outline a conceptual foundation and then use Assurance 2.0 as an engineering route to operationalize using structured argumentation, evidence, confidence, defeaters, and theory based automation. This leads to two linked artefacts: an Understanding Basis, which justifies why available understanding is sufficient for a decision, and a Personal Understanding Statement, through which participants make their grasp explicit and challengeable. We also identify risks that automation may improve artefact production while weakening understanding, and we propose initial directions for evaluating both efficacy and epistemic impact.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 2 minor

Summary. The paper claims that safety and assurance cases risk detachment from the understanding required for responsible decisions, driven by pressures of tempo, complexity, and AI-generated artefacts that may yield coherent outputs without genuine human comprehension. Drawing on Catherine Elgin's epistemology, it proposes making understanding an explicit, assessable component of decision-making via the Assurance 2.0 framework. This operationalization introduces two artefacts—an Understanding Basis justifying sufficiency of available understanding (about behavior, evidence, assumptions, risks, and uncertainty) and a Personal Understanding Statement rendering individual grasp explicit and challengeable—while identifying automation risks and outlining directions for evaluating efficacy and epistemic impact.

Significance. If the artefacts can be integrated without introducing new forms of detachment, the proposal could meaningfully reframe assurance practices in critical socio-technical systems by treating understanding as a first-class, challengeable element rather than an implicit byproduct. The work merits credit for its non-circular use of external epistemology (Elgin), explicit flagging of automation-induced epistemic risks, and call for future falsifiable evaluation of epistemic impact, which strengthens a purely conceptual contribution.

major comments (1)
  1. [Section describing the two linked artefacts (following the outline of Assurance 2.0 operationalization)] The central operational claim—that the Understanding Basis and Personal Understanding Statement render understanding 'assessable and defensible' within Assurance 2.0—rests on structured argumentation, evidence, confidence, defeaters, and theory-based automation, but the manuscript provides only high-level descriptions of these artefacts without concrete criteria for determining sufficiency or mechanisms for enforcement and challenge. This underspecification is load-bearing for the proposal's practicality.
minor comments (2)
  1. The distinction between the proposed artefacts and pre-existing Assurance 2.0 elements (e.g., how the Understanding Basis differs from or augments existing confidence/defeater structures) could be clarified with a brief comparison table or diagram to aid readers already familiar with the framework.
  2. A short illustrative example—perhaps a simplified assurance fragment showing an Understanding Basis and Personal Understanding Statement in use—would help ground the conceptual discussion without requiring full empirical validation.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for their constructive review and recommendation of minor revision. We address the concern about underspecification of the proposed artefacts below, while preserving the paper's conceptual focus on reframing understanding within assurance practices.

read point-by-point responses

  1. Referee: The central operational claim—that the Understanding Basis and Personal Understanding Statement render understanding 'assessable and defensible' within Assurance 2.0—rests on structured argumentation, evidence, confidence, defeaters, and theory-based automation, but the manuscript provides only high-level descriptions of these artefacts without concrete criteria for determining sufficiency or mechanisms for enforcement and challenge. This underspecification is load-bearing for the proposal's practicality.

    Authors: We acknowledge that the manuscript presents the Understanding Basis and Personal Understanding Statement primarily through high-level descriptions tied to the Assurance 2.0 framework, structured argumentation, evidence, confidence, defeaters, and theory-based automation. This level of detail aligns with the paper's aim as a conceptual contribution that draws on Elgin's epistemology to reframe understanding as an explicit element, rather than a fully specified methodology. To strengthen practicality, we will revise the section on the two artefacts to include illustrative examples of sufficiency criteria (such as how specific defeaters related to residual uncertainty or assumption validity could be addressed) and mechanisms for challenge (such as integration with existing review and argumentation protocols). These additions will remain illustrative and grounded in the existing outline, as comprehensive enforcement mechanisms would require further empirical development. We view this as a partial revision that directly responds to the concern without altering the paper's scope. revision: partial

Circularity Check

0 steps flagged

No significant circularity

full rationale

The paper is a purely conceptual proposal that reframes assurance practice by making 'understanding' (drawing explicitly on Catherine Elgin's external epistemology) an assessable component, then introduces two new artefacts (Understanding Basis and Personal Understanding Statement) as operational extensions of the Assurance 2.0 framework. No equations, fitted parameters, self-definitional loops, or load-bearing self-citations appear; the text presents the move as a forward-looking suggestion, flags automation risks, and explicitly defers efficacy and epistemic-impact evaluation to future work. The derivation chain therefore remains self-contained and does not reduce any central claim to its own inputs by construction.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 2 invented entities

The proposal rests on philosophical assumptions about understanding and introduces two new conceptual artefacts without independent empirical grounding or falsifiable predictions beyond the framework itself.

axioms (1)
  • domain assumption Catherine Elgin's epistemology of understanding supplies an appropriate foundation for making comprehension explicit and assessable in engineering and governance decisions.
    Invoked directly to outline the conceptual foundation for the Understanding Basis and Personal Understanding Statements.
invented entities (2)
  • Understanding Basis no independent evidence
    purpose: Justifies why available understanding is sufficient for a given decision.
    New artefact proposed to operationalize the explicit assessment of understanding.
  • Personal Understanding Statement no independent evidence
    purpose: Allows participants to make their grasp explicit and challengeable.
    New artefact proposed to support structured argumentation and defeaters.

pith-pipeline@v0.9.0 · 5479 in / 1496 out tokens · 56530 ms · 2026-05-10T19:28:28.933403+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

19 extracted references · 19 canonical work pages

  1. [1]

    John Fingleton, Nuclear Regulatory Review 2025, Enabling nuclear delivery through regulatory reform, https://assets.publishing.service.gov.uk/media/692080f75c394e481336a b89/nuclear-regulatory-review-2025.pdf

    work page 2025

  2. [2]

    Sofia Guerra and Heidy Khlaaf, Fission for Algorithms, The Undermining of Nuclear Regulation in Service of AI, AI Now Institute, Nov 2025

    work page 2025

  3. [3]

    January 2025

    CISA, DARPA, OUSD R&E, NSA, Closing the Software Understanding Gap. January 2025

    work page 2025

  4. [4]

    Sankaranarayanan, S., et al. (2026). Mitigating ‘Epistemic Debt’ in Generative AI-Scaffolded Novice Programming using Metacognitive Scripts, arXiv:2602.20206

    work page arXiv 2026

  5. [5]

    The misunderstood limits of folk science: an illusion of explanatory depth

    Rozenblit and Keil, "The misunderstood limits of folk science: an illusion of explanatory depth”, Cognitive Science, 26(5), 2002

    work page 2002

  6. [6]

    Office for Nuclear Regulation, Safety Assessment Principles for nuclear facilities, 2014 edition, revision 1 (January 2020), ONR cm9 ref 2019/367414

    work page 2014

  7. [7]

    Office for Nuclear Regulation, ONR Technical Assessment Guide (TAG) Fundamental Principles of safety assessment Issue No.: 8, April- 2023, NS-TAST-GD-004, 2023/16035

    work page 2023

  8. [8]

    Elgin, True Enough, ISBN 9780262036535, MIT Press 2017, see also http://catherineelgin.com/Understanding.html

    Catherine Z. Elgin, True Enough, ISBN 9780262036535, MIT Press 2017, see also http://catherineelgin.com/Understanding.html

    work page 2017

  9. [9]

    Assurance 2.0: A Manifesto

    Bloomfield, R., Rushby, J., “Assurance 2.0: A Manifesto”, https://doi.org/10. 48550/arXiv.2004.10474, preprint available as arXiv:2004.10474 see also the collection of Assurance 2.0 papers at https://www.csl.sri.com/users/rushby/assurance2.0

    work page arXiv 2004

  10. [10]

    The SHIP Safety Case

    Bishop, P. G. & Bloomfield, R. E. “The SHIP Safety Case”, Safecomp '95, The 14th International Conference on Computer Safety, Reliability and Security, London, UK, Springer 1995

    work page 1995

  11. [11]

    Assurance of AI Systems from a Depend- ability Perspective

    R. Bloomfield and J. Rushby, “Assurance of AI systems from a dependability perspective,” arXiv:2407.13948, updated June 2025

    work page arXiv 2025

  12. [12]

    Partisan Review, XX/4, 1954], in Essays in Understanding, 1930-1954

    Hannah Arendt, Understanding and Politics (The Difficulties of Understanding). Partisan Review, XX/4, 1954], in Essays in Understanding, 1930-1954

    work page 1954

  13. [13]

    url: https://www.csl.sri.com/ users/rushby/papers/assure24.pdf (visited on 05/08/2026)

    R. Bloomfield and J. Rushby, "Models are Central to AI Assurance," 2024 IEEE 35th International Symposium on Software Reliability Engineering Workshops (ISSREW), Tsukuba, Japan, 2024, pp. 199-202, doi: 10.1109/ISSREW63542.2024.00078

    work page doi:10.1109/issrew63542.2024.00078 2024

  14. [14]

    E. A. Lee, Plato and the Nerd: The Creative Partnership of Humans and Technology. MIT Press, 2017

    work page 2017

  15. [15]

    Every good regulator of a system must be a model of that system,

    R. C. Conant and W. R. Ashby, “Every good regulator of a system must be a model of that system,” International Journal of Systems Science, vol. 1, no. 2, pp. 89–97, 1970

    work page 1970

  16. [16]

    Charles Sanders Peirce, Stanford Encyclopaedia of Philosophy, plato.stanford.edu, accessed August 14, 2025

    work page 2025

  17. [17]

    Wiggins, Grant P., 1950– Understanding by design, and Grant Wiggins and Jay McTighe, Expanded 2nd ed. p. cm. Includes bibliographical references and index, ISBN 1-4166-0035-3

    work page 1950

  18. [18]

    Srivatsan Varadarajan et al, Enabling Theory-based Continuous Assurance: A Coherent Approach with Semantics and Automated Synthesis, SASSUR Workshop, Sept 2024, SafeComp Workshops, Springer LNCS 14989, pp 173—187

    work page 2024

  19. [19]

    Graydon and S

    M S. Graydon and S. M. Lehman, Examining Proposed Uses of LLMs to Produce or Assess Assurance Arguments, Report, National Aeronautics and Space Administration, NASA-TM-20250001849, March 2025. Preprint for Workshop on Formal Arguments for CPS Certification FACCT May 2026

    work page 2025