REVIEW 2 major objections 5 minor 90 references
AI trustworthiness can be represented as an auditable, learnable mapping from measured profiles to discrete levels.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-01 21:14 UTC pith:ST42QA45
load-bearing objection A coherent, honest governance methodology whose synthetic proof-of-concept is internally circular; the framework is plausible and worth reviewing, but the empirical claims should be reframed. the 2 major comments →
A Methodology for Auditable Trustworthiness Levels in AI Lifecycle Governance
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Governance-relative trustworthiness can be formally represented by a protocol PA of measurable dimensions, aggregated into a profile QA(t) in [0,1]^n, and mapped by an interpretable rule bT_A,N to a level in {1,...,K}. The paper argues that expert-defined anchor regions, expressed as threshold constraints on selected dimensions, are enough to build a labeled dataset, and that a shallow decision tree trained on those labels gives an auditable operational rule for post-deployment monitoring. The learned rule may preserve, compress, or refine the expert scale; in stable settings it can be discarded in favor of the expert rule. Level transitions become explicit events, and two diagnostics, bound
What carries the argument
The load-bearing object is the trustworthiness profile QA(t): a vector of dimension scores in [0,1]^n assembled under a documented protocol. Around it sit three constructs: expert-defined anchor regions R1,...,RK, which supply coarse labels; a decision tree—an interpretable threshold rule partitioning the profile space into axis-aligned regions—learned from those labels; and the two diagnostics, boundary margin (distance from the current profile to the boundary of its assigned region) and profile drift (normalized change in profile over a fixed horizon). The tree carries the argument by turning costly, sparse expert judgment into a repeatable rule; the diagnostics give that rule a monitoring
Load-bearing premise
The framework stands or falls on whether the expert-assigned reference labels genuinely capture trustworthiness and whether a rule learned from them keeps working on new deployment data; the simulations make this easy by generating labels from the same threshold rules that serve as training inputs.
What would settle it
Give independent experts real or realistic profiles and have them assign trustworthiness levels without seeing the rule; train the decision tree on their labels and evaluate out-of-time. If agreement among experts is low, if test accuracy approaches chance on held-out segments, or if boundary-margin and drift alerts do not line up with expert-identified level changes, the central generalizability claim would be refuted.
If this is right
- Design-time expert labeling can be reused as a routine monitoring rule, reducing the need to convene full expert panels for every post-deployment check.
- A level transition, a small boundary margin, or a large profile drift provides documented evidence for deciding whether a change is substantial enough to trigger reassessment.
- The governance artifact (protocol, labels, learned rule, monitoring results) makes trustworthiness assignments versionable and contestable over time.
- In stable environments with little regime variation, the methodology's honest output is to keep the expert rule and not learn; this is a governance decision, not a gap.
- When rare low-trustworthiness states are weakly represented, the learned rule compresses the expert scale and exposes that insufficiency, prompting validation or relabeling.
Where Pith is reading between the lines
- If reference labels are noisy or contested across raters, the learned rule inherits that disagreement; a natural extension would be to quantify inter-rater reliability and carry it into confidence sets around level assignments—something the paper does not do.
- The boundary margin could be used to set monitoring cadence or escalation thresholds dynamically, since small margins indicate fragility; this operationalization is left implicit.
- The same profile-and-rule layer could serve as a common reporting format across multiple deployments of the same system, but the paper's own experiments show that differing institutional labeling granularity would still prevent direct comparability.
- A probabilistic ordinal model over levels, rather than a point classifier, would give auditors an explicit uncertainty estimate and could be tested on the same synthetic traces.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a two-part methodology for AI lifecycle governance. The formal framework represents trustworthiness through a documented protocol of measurable dimensions, aggregates them into a profile Q(t), and learns an empirical trustworthiness-level rule from expert-labeled reference data using decision trees (Eq. (1)). The governance procedure embeds the learned rule in pre-deployment labeling, post-deployment monitoring, and reassessment, together with two diagnostics—boundary margin and profile drift—and a reporting artifact. A simulation study with three synthetic lifecycle experiments (two-dimensional baseline, asynchronous four-dimensional lending, and two hospital copies) illustrates the behavior of the framework. The paper is explicitly scoped as a proof of concept, not external validation.
Significance. The formal core is coherent: it cleanly separates trustworthiness protocol, profile, reference labels, and learned rule, and the decision-tree instantiation is a sensible choice for auditable threshold-based governance. The two diagnostics are simple and potentially useful for monitoring. The governance procedure is well-structured and connects to EU AI Act concepts such as conformity assessment, post-market monitoring, and substantial modification. The simulation code and data-generation equations are reported in detail, making the proof of concept reproducible. The paper is honest about the framework's dependence on label quality and protocol maturity (Section 7). If adopted by an institution with a well-documented labeling process, the framework offers a practical translation layer between high-level trustworthiness principles and measurable lifecycle monitoring.
major comments (2)
- [Section 5 and Appendix Tables 6, 9, 12] The synthetic labels are deterministic threshold functions of the same profile coordinates used as features. Consequently, the near-perfect results in Tables 7 and 10 largely recover the label-generating function. This does not test the claim in Section 3.3 that the learned rule 'extends costly expert judgment to routine post-deployment monitoring' when labels are noisy, delayed, contested, or depend on unmeasured variables. Section 7 acknowledges the general limitation, but the main text should explicitly state that the simulation uses feature-determined labels and therefore serves as an internal consistency check rather than evidence of generalization. I recommend either adding a simulation with label noise or adversarial labeling, or revising the wording in Section 3.3 and the contribution list to avoid overclaiming.
- [Section 3.3, dataset labeling procedure] The paper leaves open how the learned rule behaves in profile-space regions not covered by expert anchor regions R1,...,RK. Since the decision tree assigns a label to every profile, uncovered borderline regions are populated by interpolation/extrapolation from labeled anchor cases. The paper notes borderline cases may be escalated or excluded (footnote 5), but the operational rule will still make assignments there. In the proof of concept this issue is invisible because all profiles lie on trajectories and labels are total. Please either add a discussion of how uncovered regions should be validated (e.g., targeted expert review of tree-split boundaries) or state this as an explicit limitation of the framework.
minor comments (5)
- [Section 6.1] The text says 'the learned partition is simple.' Consider showing the actual tree rules in the main text or appendix to make this concrete and support reproducibility.
- [Table 7] For the clinical setting, balanced accuracy and macro-F1 on the training segment are lower because some trustworthiness levels are weakly represented. Please report per-class support counts so readers can calibrate these numbers.
- [Footnote 3] The reference trustworthiness-level rule T*_A is introduced but not used later. Either integrate it into the main text (e.g., when defining the learning target) or remove it to avoid a dangling concept.
- [Section 8.2, simulation equation] The noise terms are written as ε(1)(t) and ε(2)(t), while the text refers to 'noise_std' with separate standard deviations. Please make the mapping between the notation and the parameter table explicit at first use.
- [Figure 4] In the Hospital Alpha panel, the 'Operative/predicted TW level' is identical to the 'Local true TW level' by construction because no learned rule is used. The legend or caption should state this explicitly to avoid confusion.
Circularity Check
No significant circularity: the formal framework is a supervised-learning/governance translation layer, and the synthetic proof of concept explicitly disclaims external validation.
full rationale
The paper's central derivation is not circular. It defines trustworthiness protocols, profiles, and reference labels as distinct objects, then poses a standard supervised-learning problem: learn an interpretable rule h from profiles QA(t) to expert-assigned labels Yi (Eq. 1). The learned rule is an empirical approximation of the expert labeling procedure, not a redefinition of the target in terms of the model output. In the simulations, reference labels are generated by hand-set threshold rules over the same profile coordinates (Appendix Tables 6, 9, 12), so a decision tree can recover those thresholds; however, this is a self-consistency check, not a hidden reduction. The paper explicitly states: 'The aim of this study is not to establish external validity across domains, but to illustrate how the proposed governance objects behave under controlled conditions' (Section 5), and later notes 'the framework is only as informative as the trustworthiness object that an institution can actually construct and maintain' (Section 7). Experiment 3 also demonstrates that the learned rule can fail to recover the expert scale, showing the mapping is not forced by construction. Self-citations to Ferrario 2024, 2025, and 2026 are contextual and not load-bearing; the formal definitions and learning objective are stated in the paper itself. Therefore no step reduces to its own inputs by definition or by fitted-parameter renaming.
Axiom & Free-Parameter Ledger
free parameters (5)
- Expert-defined trustworthiness-level thresholds (simulation labels) =
e.g., recommender: q_acc<0.60 → level 1; q_acc≥0.60 & q_rob<0.60 → level 2; else level 3
- Lifecycle simulation coefficients (baseline b, drift d, shock/update magnitudes, event times, noise std, cadence) =
Tables 5, 8, 11
- Decision-tree hyperparameter grid =
Table 4: criterion {gini,entropy}, max_leaf_nodes {3..12,None}, max_depth {2..8,None}, ccp_alpha {0,1e-4,...,1e-1}
- Trustworthiness-level scale K and protocol dimensions =
K=3 or 4; dimensions and metrics per experiment
- Boundary-margin distance and profile-drift horizon h =
dist unspecified; h=7 (dense) or 1 (sparse)
axioms (5)
- domain assumption Trustworthiness can be treated as objective relative to a documented governance standard and summarized by a finite profile of dimensions normalized to [0,1].
- domain assumption Expert reference labels Yi from a documented adjudication procedure are reliable ground truth for learning.
- ad hoc to paper Axis-parallel threshold rules (decision trees) are an adequate model class for governance trustworthiness levels.
- ad hoc to paper The synthetic generation equation z(t)=clip[0,1](b-dτ+ε1+Σs_j 1 + Σu_k 1 + ε2) captures lifecycle dynamics relevant to trustworthiness.
- standard math Temporal validation and standard decision-tree learning assumptions (stable target, informative labels) hold.
read the original abstract
AI governance increasingly requires judgments about whether an AI system remains adequately trustworthy over time, whether observed changes are tolerable, and how such judgments should be documented in a transparent and contestable way. Yet existing work on AI trustworthiness remains either too high-level to support lifecycle monitoring and reassessment or too narrowly metric-driven to connect with governance needs. We therefore propose a lightweight methodology for auditable trustworthiness levels in AI governance. The methodology has two components: a formal framework for representing and learning trustworthiness levels, and a lightweight AI lifecycle governance procedure for documenting, monitoring, and reassessing them over time. The formal framework models governance-relative trustworthiness through a context-sensitive protocol of measurable dimensions and learns trustworthiness levels as interpretable rules over trustworthiness profiles. Using decision trees as an interpretable proof-of-concept model class, the methodology yields explicit trustworthiness plateaus, readable level transitions, and two simple lifecycle diagnostics: boundary margins and profile drift. The governance procedure embeds these formal objects in a conformity-oriented workflow for design-time labeling, post-deployment monitoring, reassessment, and reporting. It also assigns human responsibilities and control gates for protocol design, validation, monitoring, and reassessment. We illustrate the methodology on synthetic AI lifecycle traces involving degradation, shocks, updates, heterogeneous monitoring cadences, and system comparison. Our methodology does not replace legal or other expert judgment: it supports conformity documentation and lifecycle monitoring by providing an evidential basis for documenting and tracking AI governance-relevant changes over time.
Figures
Reference graph
Works this paper leans on
-
[1]
2016 , howpublished =
2016
-
[2]
2019 , month = apr, institution =
2019
-
[3]
Recommendation on the Ethics of Artificial Intelligence , year =
-
[4]
Conformity assessment under the
Thelisson, Eva and Verma, Himanshu , journal=. Conformity assessment under the. 2024 , publisher=
2024
-
[5]
Conformity assessments and post-market monitoring:
M. Conformity assessments and post-market monitoring:. Minds and Machines , volume=. 2022 , publisher=
2022
-
[6]
Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency , pages=
Nolte, Henrik and Rateike, Miriam and Finck, Mich. Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency , pages=
2025
-
[7]
2019 , organization=
Towards an ethical code for data-based business , author=. 2019 , organization=
2019
-
[8]
The law of recitals in European Community legislation , author=. ILSA J. Int'l & Comp. L. , volume=. 2008 , publisher=
2008
-
[9]
Trustworthy artificial intelligence and the
Laux, Johann and Wachter, Sandra and Mittelstadt, Brent , journal=. Trustworthy artificial intelligence and the. 2024 , publisher=
2024
-
[10]
Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT* '19) , year =
Margaret Mitchell and Simone Wu and Andrew Zaldivar and Parker Barnes and Lucy Vasserman and Ben Hutchinson and Elena Spitzer and Inioluwa Deborah Raji and Timnit Gebru , title =. Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT* '19) , year =
-
[11]
Regulating the undefined: Addressing systemic risks in the Digital Services Act (with an appendix on the
Loi, Michele and Fabbri, Matteo and Ferrario, Andrea , journal=. Regulating the undefined: Addressing systemic risks in the Digital Services Act (with an appendix on the. 2025 , publisher=
2025
-
[12]
Nature , volume=
Role play with large language models , author=. Nature , volume=. 2023 , publisher=
2023
-
[13]
Communications of the ACM , volume=
Talking about large language models , author=. Communications of the ACM , volume=. 2024 , publisher=
2024
-
[14]
2024 , month = jun, institution =
2024
-
[15]
2022 , url =
European declaration on. 2022 , url =
2022
-
[16]
2001 , month = nov, howpublished =
2001
-
[17]
Mitigating the adverse effects of
W. Mitigating the adverse effects of. Global Business and Organizational Excellence , volume=. 2024 , publisher=
2024
-
[18]
2020 , month = feb, url =
2020
-
[19]
2017 , month = apr, url =
2017
-
[20]
2024 , publisher=
Anthropomorphising machines and computerising minds: The crosswiring of languages between Artificial Intelligence and Brain & Cognitive Sciences , author=. 2024 , publisher=
2024
-
[21]
Quine, W. V. O. , title =. 1969 , address =
1969
-
[22]
2001 , address =
Wiggins, David , title =. 2001 , address =
2001
-
[23]
Lowe, E. J. , title =. Artefact Kinds: Ontology and the Human-Made World , editor =. 2014 , pages =
2014
-
[24]
, title =
Pinter, Charles C. , title =. 2014 , publisher =
2014
-
[25]
, title =
Olson, Eric T. , title =. 2024 , publisher =
2024
-
[26]
2020 , publisher=
In AI we trust: Ethics, artificial intelligence, and reliability , author=. 2020 , publisher=
2020
-
[27]
Akbulut, Canfer and Weidinger, Laura and Manzini, Arianna and Gabriel, Iason and Rieser, Verena , booktitle=
-
[28]
Formalizing trust in artificial intelligence:
Jacovi, Alon and Marasovi. Formalizing trust in artificial intelligence:
-
[29]
2020 , address =
Bradford, Anu , title =. 2020 , address =
2020
-
[30]
2020 , address =
Petit, Nicolas , title =. 2020 , address =
2020
-
[31]
Proposed
Kazim, Emre and G. Proposed. AI and Ethics , volume=. 2023 , publisher=
2023
-
[32]
Floridi, Luciano , journal=. The. 2021 , publisher=
2021
-
[33]
Ho, Calvin Wai-Loon and Caals, Karel , journal=. How the. 2024 , publisher=
2024
-
[34]
W. The. AI and Ethics , volume=. 2025 , publisher=
2025
-
[35]
Institutionalised distrust and human oversight of artificial intelligence: towards a democratic design of
Laux, Johann , journal=. Institutionalised distrust and human oversight of artificial intelligence: towards a democratic design of. 2024 , publisher=
2024
-
[36]
A blueprint for auditing generative
M. A blueprint for auditing generative. 2025 , publisher=
2025
-
[37]
Auditing large language models:
M. Auditing large language models:. 2024 , publisher=
2024
-
[38]
Smuha, Nathalie A and Rengers, Emma and Harkens, Adam and Li, Wenlong and MacLaren, James and Piselli, Riccardo and Yeung, Karen , journal =. How the. 2021 , month = aug, doi =
2021
-
[39]
Twenty-four years of empirical research on trust in
Benk, Michaela and Kerstan, Sophie and von Wangenheim, Florian and Ferrario, Andrea , journal=. Twenty-four years of empirical research on trust in. 2025 , publisher=
2025
-
[40]
An overview of key trustworthiness attributes and
Mattioli, Juliette and Sohier, Henri and Delaborde, Agn. An overview of key trustworthiness attributes and. AI and Ethics , volume=. 2024 , publisher=
2024
-
[41]
Trustworthy artificial intelligence:
Kaur, Davinder and Uslu, Suleyman and Rittichier, Kaley J and Durresi, Arjan , journal=. Trustworthy artificial intelligence:. 2022 , publisher=
2022
-
[42]
Ala-Pietil. The. 2020 , publisher=
2020
-
[43]
Z-Inspection
Zicari, Roberto V and Brodersen, John and Brusseau, James and D. Z-Inspection. IEEE Transactions on Technology and Society , volume=. 2021 , publisher=
2021
-
[44]
Justifying our credences in the trustworthiness of
Ferrario, Andrea , journal=. Justifying our credences in the trustworthiness of. 2024 , publisher=
2024
-
[45]
Trustworthy
Li, Bo and Qi, Peng and Liu, Bo and Di, Shuai and Liu, Jingen and Pei, Jiquan and Yi, Jinfeng and Zhou, Bowen , journal=. Trustworthy. 2023 , publisher=
2023
-
[46]
2019 , howpublished =
Ethics guidelines for. 2019 , howpublished =
2019
-
[47]
and Davis, James H
Mayer, Roger C. and Davis, James H. and Schoorman, F. David , title =. Academy of Management Review , volume =. 1995 , doi =
1995
-
[48]
Ethics-based auditing to develop trustworthy
M. Ethics-based auditing to develop trustworthy. Minds and Machines , volume=. 2021 , publisher=
2021
-
[49]
Trust, distrust and commitment , author=. No. 2014 , publisher=
2014
-
[50]
Synthese , volume=
The fine-grained metaphysics of artifactual and biological functional kinds , author=. Synthese , volume=. 2009 , publisher=
2009
-
[51]
A Trustworthiness-based metaphysics of
Ferrario, Andrea , booktitle=. A Trustworthiness-based metaphysics of
-
[52]
Philosophical Explorations , volume=
The ontology of artifacts , author=. Philosophical Explorations , volume=. 2004 , publisher=
2004
-
[53]
, year =
Gallois, A. , year =
-
[54]
Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society , volume=
Social misattributions in conversations with large language models , author=. Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society , volume=
-
[55]
Ascribing functions to technical artefacts:
Vermaas, Pieter E and Houkes, Wybo , journal=. Ascribing functions to technical artefacts:. 2003 , publisher=
2003
-
[56]
Towards a science of
Rabanser, Stephan and Kapoor, Sayash and Kirgis, Peter and Liu, Kangheng and Utpala, Saiteja and Narayanan, Arvind , journal=. Towards a science of
-
[57]
2020 , note =
Assessment List for Trustworthy Artificial Intelligence (. 2020 , note =
2020
-
[58]
Frontiers in Artificial Intelligence , year =
Radclyffe, Catherine and others , title =. Frontiers in Artificial Intelligence , year =
-
[59]
2023 , number =
Artificial Intelligence Risk Management Framework (. 2023 , number =
2023
-
[60]
Proceedings of the Conference on Fairness, Accountability, and Transparency , year =
Mitchell, Margaret and Wu, Simone and Zaldivar, Andrew and Barnes, Parker and Vasserman, Lucy and Hutchinson, Ben and Spitzer, Elena and Raji, Inioluwa Deborah and Gebru, Timnit , title =. Proceedings of the Conference on Fairness, Accountability, and Transparency , year =
-
[61]
Datasheets for Datasets , journal =
Gebru, Timnit and Morgenstern, Jamie and Vecchione, Briana and Vaughan, Jennifer Wortman and Wallach, Hanna and Daum. Datasheets for Datasets , journal =. 2021 , volume =
2021
-
[62]
Bellamy, Rachel K. E. and Dey, Kuntal and Hind, Michael and Hoffman, Samuel C. and Houde, Stephanie and Kannan, Kalapriya and Lohia, Praveen and Martino, Jacqueline and Mehta, Sameep and Mojsilovi. IEEE International Conference on Data Mining Workshops , year =
-
[63]
Fairlearn: A Toolkit for Assessing and Improving Fairness in
Bird, Sarah and Dud. Fairlearn: A Toolkit for Assessing and Improving Fairness in. Microsoft Research White Paper , year =
-
[64]
Arya, Vijay and Bellamy, Rachel K. E. and Chen, Pin-Yu and Dhurandhar, Amit and Hind, Michael and Hoffman, Samuel C. and Houde, Stephanie and Liao, Q. Vera and Luss, Ronny and Mojsilovi. One Explanation Does Not Fit All: A Toolkit and Taxonomy of. Proceedings of the IJCAI 2019 Workshop on Explainable Artificial Intelligence , year =
2019
-
[65]
Zicari, Roberto V. and others , title =. arXiv preprint arXiv:2206.09887 , year =
-
[66]
and others , title =
Zicari, Roberto V. and others , title =. IEEE International Conference on Technology and Society , year =
-
[67]
2021 , number =
Tools for Trustworthy AI: A Framework to Compare Implementation Tools for Trustworthy AI Systems , institution =. 2021 , number =
2021
-
[68]
Proceedings of the Workshop on AI Governance , year =
McCormack, Laura and others , title =. Proceedings of the Workshop on AI Governance , year =
-
[69]
and others , title =
Schlicker, N. and others , title =. Computers in Human Behavior , year =
-
[70]
arXiv preprint arXiv:2602.16666 , year =
Rabanser, Stephan and Kapoor, Sayash and Kirgis, Peter and Liu, Kangheng and Utpala, Saiteja and Narayanan, Arvind , title =. arXiv preprint arXiv:2602.16666 , year =
-
[71]
arXiv preprint arXiv:2603.16586 , year =
Kaptein, Maurits and Khan, Vassilis-Javed and Podstavnychy, Andriy , title =. arXiv preprint arXiv:2603.16586 , year =
-
[72]
Pinar and Casper, Stephen and Kolt, Noam , title =
Staufer, Leon and Feng, Kevin and Wei, Kevin and Bailey, Luke and Duan, Yawen and Yang, Mick and Ozisik, A. Pinar and Casper, Stephen and Kolt, Noam , title =. arXiv preprint arXiv:2602.17753 , year =
-
[73]
Machine learning operations (
Kreuzberger, Dominik and K. Machine learning operations (. IEEE Access , volume=. 2023 , publisher=
2023
-
[74]
Journal of Systems and Software , volume=
Adaptive data quality scoring operations framework using drift-aware mechanism for industrial applications , author=. Journal of Systems and Software , volume=. 2024 , publisher=
2024
-
[75]
A multivocal review of
Eken, Beyza and Pallewatta, Samodha and Tran, Nguyen and Tosun, Ayse and Babar, Muhammad Ali , journal=. A multivocal review of. 2025 , publisher=
2025
-
[76]
Towards a better understanding of evaluating trustworthiness in
Kemmerzell, Nils and Schreiner, Annika and Khalid, Haroon and Schalk, Michael and Bordoli, Letizia , journal=. Towards a better understanding of evaluating trustworthiness in. 2025 , publisher=
2025
-
[77]
Creating Characteristically Auditable Agentic
Phiri, Charles Chimwemwe , booktitle=. Creating Characteristically Auditable Agentic
-
[78]
Understanding
Mehrotra, Siddharth and Huang, Jin and Fu, Xuelong and Dobbe, Roel and S. Understanding. Journal of Artificial Intelligence Research , volume=
-
[79]
Nature Communications , volume=
Active label cleaning for improved dataset quality under resource constraints , author=. Nature Communications , volume=. 2022 , publisher=
2022
-
[80]
2019 , month=jun # " 11", publisher=
Structured labeling to facilitate concept evolution in machine learning , author=. 2019 , month=jun # " 11", publisher=
2019
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.