Explainable and Human-Grounded AI for Decision Support Systems: The Theory of Epistemic Quasi-Partnerships

John Dorsch; Maximilian Moll

arxiv: 2409.14839 · v1 · pith:T2GINBNMnew · submitted 2024-09-23 · 💻 cs.AI · cs.ET· cs.HC

Explainable and Human-Grounded AI for Decision Support Systems: The Theory of Epistemic Quasi-Partnerships

John Dorsch , Maximilian Moll This is my paper

Pith reviewed 2026-05-23 20:46 UTC · model grok-4.3

classification 💻 cs.AI cs.ETcs.HC

keywords explainable AIepistemic quasi-partnershipsdecision support systemshuman-grounded explanationsRCC approachtrustworthinesscounterfactual explanationsXAI

0 comments

The pith

The theory of epistemic quasi-partnerships explains XAI evidence and requires AI decision support systems to deliver reasons, counterfactuals, and confidence.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper surveys empirical work on explanation techniques such as LIME, SHAP, and Anchors, focusing on how they shape users' sense of model trustworthiness and their decision accuracy. It shows that prior accounts of suitable human-grounded reasons leave key patterns in the data unexplained and supply incomplete ethical direction for system builders. The authors therefore introduce the theory of epistemic quasi-partnerships, which treats the human-AI relationship as a joint epistemic endeavor. Under this theory the observed effects of different explanation methods fall into place, ethical requirements become clearer, and the need for three specific explanation types follows directly.

Core claim

The central claim is that the theory of epistemic quasi-partnerships correctly models the interaction between human decision-makers and AI decision support systems. The theory accounts for existing findings on explanation methods, trustworthiness, and accuracy; supplies sound ethical guidance; and shows that systems must supply reasons, counterfactuals, and confidence.

What carries the argument

The theory of epistemic quasi-partnerships, a model of human-machine interaction in which the AI supplies human-grounded epistemic support to enable joint decision making.

If this is right

AI-DSS should adopt the RCC approach to satisfy both explanatory and ethical demands.
Explanation methods such as LIME, SHAP, and Anchors must be assessed for how well they supply reasons, counterfactuals, and confidence.
Perceived trustworthiness and end-user accuracy improve when systems follow the epistemic quasi-partnership framing.
Ethical development of AI-DSS follows from treating the user-AI relation as a quasi-partnership rather than from competing accounts of explanation.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The framework could be applied to AI systems outside decision support, such as recommendation engines or planning tools.
Developers might create concrete checklists that verify whether a given explanation set covers all three RCC components.
The theory suggests new experiments that isolate the contribution of each explanation type to joint decision quality.

Load-bearing premise

Existing theories about what constitutes good human-grounded reasons either fail to explain the reviewed empirical evidence or fail to offer sound ethical advice for AI-DSS development.

What would settle it

A controlled study in which an AI-DSS using only one explanation type produces user accuracy and trustworthiness scores equal to or higher than the RCC combination, without ethical violations, would undermine the claim that all three types are required.

read the original abstract

In the context of AI decision support systems (AI-DSS), we argue that meeting the demands of ethical and explainable AI (XAI) is about developing AI-DSS to provide human decision-makers with three types of human-grounded explanations: reasons, counterfactuals, and confidence, an approach we refer to as the RCC approach. We begin by reviewing current empirical XAI literature that investigates the relationship between various methods for generating model explanations (e.g., LIME, SHAP, Anchors), the perceived trustworthiness of the model, and end-user accuracy. We demonstrate how current theories about what constitutes good human-grounded reasons either do not adequately explain this evidence or do not offer sound ethical advice for development. Thus, we offer a novel theory of human-machine interaction: the theory of epistemic quasi-partnerships (EQP). Finally, we motivate adopting EQP and demonstrate how it explains the empirical evidence, offers sound ethical advice, and entails adopting the RCC approach.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

The paper introduces EQP as a new lens for XAI in decision support and derives the RCC recommendation from it, but the case that prior theories fail to explain the evidence rests on assertion rather than explicit mapping.

read the letter

The main contribution is the theory of epistemic quasi-partnerships, which the authors position as explaining why certain explanation methods affect user trust and accuracy in AI-DSS, and which leads them to recommend supplying reasons, counterfactuals, and confidence estimates. They review empirical work on LIME, SHAP, and Anchors and tie it to trustworthiness and end-user performance, which gives the argument a concrete starting point. That review is the part that could be useful to others working on human-grounded explanations. The soft spot is the central claim that existing theories about good human-grounded reasons either fail to account for the evidence or fail to give sound ethical advice. The abstract states this is shown through the review, yet supplies no named theories, no direct contrast of their commitments against specific results, and no explicit criterion for adequacy. Without that step, the novelty of EQP and its entailment of RCC remain hard to assess; the risk that this is mostly re-labeling is not ruled out. There is no new data, no formal derivation, and no quantitative test, so the paper stays interpretive. The stress-test note is on target here: the unshown failure of predecessors carries a lot of weight. This is aimed at people in XAI ethics and design for medicine, law, or finance who want a unified ethical-explanatory standard. It deserves peer review because the topic matters and the attempt to connect evidence to normative advice is worth refining, even if the authors will need to add the missing comparisons to make the argument hold.

Referee Report

2 major / 2 minor

Summary. The paper claims that ethical and explainable AI for decision support systems requires the RCC approach (reasons, counterfactuals, and confidence). It reviews empirical studies on LIME/SHAP/Anchors and their effects on trustworthiness and user accuracy, asserts that existing theories of human-grounded reasons neither adequately explain this evidence nor provide sound ethical advice, introduces the theory of epistemic quasi-partnerships (EQP) as a novel framework for human-machine interaction, and argues that EQP explains the reviewed evidence, supplies ethical guidance, and entails adopting RCC.

Significance. If the central interpretive claim holds—that prior theories fail where EQP succeeds—the work would supply a new organizing framework for XAI in DSS that links empirical findings to ethical prescriptions. However, because the manuscript offers no new data, formal derivation, or quantitative validation and rests entirely on an interpretive literature review, its significance remains conditional on whether the asserted failure of predecessors is convincingly demonstrated.

major comments (2)

[Abstract] Abstract and opening sections: the load-bearing assertion that 'current theories about what constitutes good human-grounded reasons either do not adequately explain this evidence or do not offer sound ethical advice' is stated but not executed. No specific prior theories are named, no table or section maps theoretical commitments to particular empirical results (e.g., LIME trustworthiness studies), and no explicit criterion for 'adequate explanation' or 'sound ethical advice' is supplied. Because EQP's novelty and its entailment of RCC are positioned as filling exactly this gap, the unshown comparative failure is central to the argument.
[Theory of Epistemic Quasi-Partnerships] The manuscript presents EQP as explaining the empirical evidence on explanation methods, trustworthiness, and accuracy, yet supplies no explicit derivation or mapping showing how EQP's axioms generate the observed patterns (e.g., why certain explanation methods increase perceived trustworthiness while others do not). Without this step, the claim that EQP 'explains the evidence' remains interpretive rather than demonstrated.

minor comments (2)

[Abstract] The abstract and introduction would benefit from a short table listing the reviewed empirical studies, the explanation methods tested, and the key findings on trustworthiness/accuracy to make the evidence base transparent.
Notation for the RCC components and EQP constructs should be introduced consistently and defined on first use to aid readability for readers outside the immediate subfield.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive comments, which identify opportunities to strengthen the execution of the paper's central interpretive claims. We address each major comment below and commit to revisions that make the comparative analysis and mappings more explicit while preserving the manuscript's interpretive character.

read point-by-point responses

Referee: [Abstract] Abstract and opening sections: the load-bearing assertion that 'current theories about what constitutes good human-grounded reasons either do not adequately explain this evidence or do not offer sound ethical advice' is stated but not executed. No specific prior theories are named, no table or section maps theoretical commitments to particular empirical results (e.g., LIME trustworthiness studies), and no explicit criterion for 'adequate explanation' or 'sound ethical advice' is supplied. Because EQP's novelty and its entailment of RCC are positioned as filling exactly this gap, the unshown comparative failure is central to the argument.

Authors: We agree that the manuscript would benefit from a more explicit comparative analysis. In the revised version we will add a new subsection (provisionally titled 'Gaps in Existing Accounts of Human-Grounded Reasons') that (1) names representative prior theories (e.g., fidelity-based accounts, cognitive-load models of explanation, and ethical frameworks from the XAI literature such as those emphasizing transparency or user autonomy), (2) includes a mapping table that links each theory's core commitments to specific empirical outcomes reported in the LIME/SHAP/Anchors studies reviewed in the paper, and (3) states explicit criteria for 'adequate explanation' (consistency with observed changes in user accuracy and trustworthiness) and 'sound ethical advice' (compatibility with principles of epistemic responsibility). These additions will directly support the claim that existing theories fall short and will clarify how EQP addresses the identified gap. revision: yes
Referee: [Theory of Epistemic Quasi-Partnerships] The manuscript presents EQP as explaining the empirical evidence on explanation methods, trustworthiness, and accuracy, yet supplies no explicit derivation or mapping showing how EQP's axioms generate the observed patterns (e.g., why certain explanation methods increase perceived trustworthiness while others do not). Without this step, the claim that EQP 'explains the evidence' remains interpretive rather than demonstrated.

Authors: The referee is correct that the current text leaves the connection between EQP axioms and empirical patterns largely implicit. While EQP is offered as a normative philosophical framework rather than a formal predictive model, we will add a dedicated subsection that supplies an explicit logical mapping. This subsection will articulate how the central EQP axioms (shared epistemic goals, mutual accountability, and the quasi-partnership relation) generate expectations about which explanation styles (e.g., high-precision rule-based outputs versus feature-importance scores) are likely to increase perceived trustworthiness or accuracy in the reviewed studies. The mapping will be presented as a structured argument with illustrative references to the LIME, SHAP, and Anchors findings already discussed in the manuscript. revision: partial

Circularity Check

0 steps flagged

No significant circularity in the derivation chain

full rationale

The paper's chain runs from a review of independent external empirical studies (LIME/SHAP/Anchors on trustworthiness and accuracy) to an assessment that unnamed prior theories fail to explain those results or supply ethical guidance, to the introduction of EQP as a new framework, to the application of EQP back to the same evidence plus derivation of the RCC approach. This structure does not exhibit any of the enumerated circularity patterns: there are no equations, no fitted parameters renamed as predictions, no self-citations used as load-bearing uniqueness theorems, and no ansatz or renaming that reduces the central claim to its own inputs by construction. The empirical literature cited is external to the paper, and the proposal of EQP is presented as a standard theoretical response to a claimed explanatory gap rather than a self-referential loop.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 1 invented entities

The paper rests on domain assumptions about the state of empirical XAI evidence and the inadequacy of prior theories; it introduces the EQP construct as a new theoretical entity without independent empirical grounding.

axioms (2)

domain assumption Current empirical XAI literature demonstrates specific relationships between explanation methods, perceived trustworthiness, and end-user accuracy.
Invoked to establish the evidence base that existing theories fail to explain.
ad hoc to paper Existing theories about good human-grounded reasons neither adequately explain the evidence nor offer sound ethical advice.
This premise directly motivates the need for EQP.

invented entities (1)

Epistemic quasi-partnerships (EQP) no independent evidence
purpose: New theory of human-machine interaction that frames AI-DSS as epistemic partners and derives the RCC explanation requirements.
Postulated as the central novel construct; no independent falsifiable evidence provided in the abstract.

pith-pipeline@v0.9.0 · 5707 in / 1478 out tokens · 41473 ms · 2026-05-23T20:46:14.740191+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

6 extracted references · 6 canonical work pages · 2 internal anchors

[1]

Amazon Web Services. (2024). Model explainability with Amazon Web Services Artificial Intelligence and Machine Learning Solutions [White Paper]. Amazon Web Services, Inc. https://docs.aws.amazon.com/pdfs/whitepapers/latest/model-explainability-aws-ai-ml/model-explainability-aws-ai- ml.pdf#interpretability-versus-explainability Bansal, G., Nushi, B., Kamar...

work page doi:10.1609/hcomp.v7i1.5285 2024
[2]

Quasi-Metacognitive Machines: Why We Don’t Need Morally Trustworthy AI and Communicating Reliability is Enough

https://doi.org/10.1145/3610219 Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015). Algorithm aversion: People erroneously avoid algorithms after seeing them err. Journal of Experimental Psychology: General, 144(1), 114-126. https://doi.org/10.1037/xge0000033 Dorsch, J., Deroy, O. “Quasi-Metacognitive Machines: Why We Don’t Need Morally Trustworthy AI ...

work page doi:10.1145/3610219 2015
[3]

Explainable Ai

https://doi.org/10.1145/3432934 Goldman, A. I. (1991). Knowledge in a Social World. Oxford University Press. Hardwig, J. (1985). Epistemic dependence. Journal of Philosophy, 82(7), 335-349. https://www.jstor.org/stable/2026523 Hase, P., & Bansal, M. (2020). Evaluating explainable AI: Which algorithmic explanations help users predict model behavior? arXiv ...

work page doi:10.1145/3432934 1991
[4]

https://doi.org/10.3390/s23042013 Miller, T., Howe, P., & Sonenberg, L. (2017). Explainable AI: Beware of inmates running the asylum or: How I learnt to stop worrying and love the social and behavioural sciences. arXiv preprint arXiv:1712.00547. https://doi.org/10.48550/arXiv.1712.00547 Miller, T. (2019). Explanation in artificial intelligence: Insights f...

work page internal anchor Pith review Pith/arXiv arXiv doi:10.3390/s23042013 2017
[5]

T., Singh, S., & Guestrin, C

https://doi.org/10.1145/3583558 Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). Why should I trust you?" Explaining the predictions of any classifier. Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining . https://doi.org/10.1145/2939672.2939778 Ribeiro, M. T., Singh, S., & Guestrin, C. (2018). Anchors: High...

work page doi:10.1145/3583558 2016
[6]

A Human-Grounded Evaluation of SHAP for Alert Processing

https://doi.org/10.1145/3479515 Weerts, H. J., van Ipenburg, W., & Pechenizkiy, M. (2019). A Human-Grounded Evaluation of SHAP for Alert Processing. arXiv preprint arXiv:1907.03324 https://doi.org/10.48550/arXiv.1907.03324 Zieglmeier, V., & Lehene, A. M. (2022). Designing Trustworthy User Interfaces. Proceedings of the 33rd Australian Conference on Human-...

work page internal anchor Pith review Pith/arXiv arXiv doi:10.1145/3479515 2019

[1] [1]

Amazon Web Services. (2024). Model explainability with Amazon Web Services Artificial Intelligence and Machine Learning Solutions [White Paper]. Amazon Web Services, Inc. https://docs.aws.amazon.com/pdfs/whitepapers/latest/model-explainability-aws-ai-ml/model-explainability-aws-ai- ml.pdf#interpretability-versus-explainability Bansal, G., Nushi, B., Kamar...

work page doi:10.1609/hcomp.v7i1.5285 2024

[2] [2]

Quasi-Metacognitive Machines: Why We Don’t Need Morally Trustworthy AI and Communicating Reliability is Enough

https://doi.org/10.1145/3610219 Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015). Algorithm aversion: People erroneously avoid algorithms after seeing them err. Journal of Experimental Psychology: General, 144(1), 114-126. https://doi.org/10.1037/xge0000033 Dorsch, J., Deroy, O. “Quasi-Metacognitive Machines: Why We Don’t Need Morally Trustworthy AI ...

work page doi:10.1145/3610219 2015

[3] [3]

Explainable Ai

https://doi.org/10.1145/3432934 Goldman, A. I. (1991). Knowledge in a Social World. Oxford University Press. Hardwig, J. (1985). Epistemic dependence. Journal of Philosophy, 82(7), 335-349. https://www.jstor.org/stable/2026523 Hase, P., & Bansal, M. (2020). Evaluating explainable AI: Which algorithmic explanations help users predict model behavior? arXiv ...

work page doi:10.1145/3432934 1991

[4] [4]

https://doi.org/10.3390/s23042013 Miller, T., Howe, P., & Sonenberg, L. (2017). Explainable AI: Beware of inmates running the asylum or: How I learnt to stop worrying and love the social and behavioural sciences. arXiv preprint arXiv:1712.00547. https://doi.org/10.48550/arXiv.1712.00547 Miller, T. (2019). Explanation in artificial intelligence: Insights f...

work page internal anchor Pith review Pith/arXiv arXiv doi:10.3390/s23042013 2017

[5] [5]

T., Singh, S., & Guestrin, C

https://doi.org/10.1145/3583558 Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). Why should I trust you?" Explaining the predictions of any classifier. Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining . https://doi.org/10.1145/2939672.2939778 Ribeiro, M. T., Singh, S., & Guestrin, C. (2018). Anchors: High...

work page doi:10.1145/3583558 2016

[6] [6]

A Human-Grounded Evaluation of SHAP for Alert Processing

https://doi.org/10.1145/3479515 Weerts, H. J., van Ipenburg, W., & Pechenizkiy, M. (2019). A Human-Grounded Evaluation of SHAP for Alert Processing. arXiv preprint arXiv:1907.03324 https://doi.org/10.48550/arXiv.1907.03324 Zieglmeier, V., & Lehene, A. M. (2022). Designing Trustworthy User Interfaces. Proceedings of the 33rd Australian Conference on Human-...

work page internal anchor Pith review Pith/arXiv arXiv doi:10.1145/3479515 2019