Pith. sign in

REVIEW 4 major objections 5 minor 17 references

Against Explainable Artificial Intelligence In Law: Why Justifiable Ai Matters. A Credit Scoring Example

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper argues that technical explainability fails consumers in credit scoring; only a broad right to explanation as legal justification—justifiable AI—can make the right operative.

desk verdict A solid legal synthesis arguing for a broad right to justification in EU credit scoring, but the 'only path' claim outruns the thin empirical evidence. read the letter →

arxiv 2608.07452 v1 pith:UIG2WFGZ submitted 2026-08-07 cs.CY

{L}ukasz Górski This is my paper · ORCID
classification cs.CY
keywords ExplainableAIXAIcreditscoringrighttoexplanationjustifiableEUconsumerlawautomateddecision-makingregulatoryfiction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the technical explanations produced by explainable-artificial-intelligence tools do not, and cannot, fulfil the legal right to explanation for consumers whose credit applications are decided by machine-learning systems. In a business-to-consumer setting such as credit scoring, the author contends, presenting model weights, formulas, or saliency maps to laypeople delivers no actionable knowledge and amounts to 'regulatory fiction'—formal compliance without substantive protection. Drawing on EU law and on empirical studies of how lawyers and laypeople actually receive explanation outputs, the paper proposes that the right to explanation should be read broadly, as a requirement of legal justification grounded in procedural, legal, and social norms. The author names this standard 'justifiable AI' and argues it is the only one that lets consumers understand and contest automated credit decisions. The credit-scoring example is offered as the first concrete application of this justifiability standard.

What carries the argument

The load-bearing distinction is between the context of discovery and the context of justification, borrowed from philosophy of science and legal realism. The paper defines 'justifiable AI' as the standard under which a decision is presented not through model internals but through a persuasive, legally grounded argumentative chain—like a court judgment that gives reasons rather than a trace of the judge's mental processes. The machinery doing the legal work is Art. 18(8) of the 2023 Consumer Credit Directive, read together with Art. 86 of the AI Act and Art. 22 GDPR, which together require a clear, comprehensible explanation of the logic, significance, and effects of automated creditworthiness assessment; the author argues these provisions are only satisfied when the explanation gives the consumer actionable grounds for contestation. On the technical side, the machinery includes the empirical finding that explanation-generation methods are built for developers and are unintelligible to lay users, which is what turns the legal requirement into a fiction.

What would settle it

A controlled study comparing how well credit applicants can formulate a challenge after receiving either a feature-weight explanation of the model's decision or a written procedural justification for the same rejection: if applicants given the technical explanation contest decisions just as effectively as those given the justification, the paper's central claim is refuted.

Watch

Extended reading notes

Core claim

The central claim is that explainability and justifiability are different things, and the law has been relying on the wrong one. Explainability addresses the context of discovery—the internal, often mathematical process by which a model produces an output—whereas justifiability addresses the context of justification, the externally grounded reasons, rooted in legal, procedural, and social norms, that make a decision defensible to the person it affects. In business-to-consumer credit scoring, the author argues, handing consumers technical explanations such as feature weights or saliency maps neither matches how humans justify decisions nor gives consumers the knowledge they need to exercise their rights, so it constitutes regulatory fiction. A broad interpretation of the right to explanation—encompassing a legal justification that sets out the categories of data used, why they are pertinent, how any profile was built, and how the profile bears on this particular decision—is therefore the only reading that makes the right operative. The paper answers its three research questions by concluding that current explanation methods are developer-oriented and fail consumers, that the right to explanation must be interpreted broadly in the existing legal framework, and that justifiability should replace technical explainability as the standard.

Load-bearing premise

The argument rests on the premise that the legal point of an explanation is to give the consumer usable grounds to challenge the decision, and that no technical description of how the model works can provide those grounds; if the right to explanation is really about transparency or auditability, or if technical explanations ever become genuinely usable by ordinary people, the case for justifiability loses its footing.

Editorial extensions

If this is right

  • Lenders would satisfy their explanation duties under Art. 18(8) CCD by supplying procedural justifications—categories of data, why they are pertinent, how a profile is built, and its role in the specific decision—rather than exposing model weights or code.
  • Regulators and courts would treat developer-oriented explainability outputs as non-compliant in business-to-consumer contexts, since they do not confer actionable knowledge on consumers.
  • The derogation in Art. 86(3) of the AI Act means the CCD's broader explanation right governs credit scoring, so recent case law on automated decisions supports an expansive reading of that right.
  • Consumers' right to present their point of view and contest the decision (Art. 18(8)(c) CCD) becomes the test of adequacy: an explanation is legally sufficient only if it equips the consumer to challenge the outcome.
  • Generating justifications with large language models, constrained by a logical rule layer encoding expert knowledge, is presented as the paper's concrete route to operationalizing justifiability.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same justifiability standard would naturally extend beyond credit scoring to any high-stakes consumer decision made by AI, such as hiring, insurance pricing, or benefit eligibility, shifting the regulatory demand from model transparency to contestable reasons.
  • If regulators adopt this standard, the trade-secrecy conflict softens: firms can protect model internals while disclosing the procedural rationale, so the legal battle moves from opening the black box to judging whether the stated reasons are adequate.
  • A testable extension follows: studies comparing consumer contestation after technical explanations versus procedural justifications should show that only the latter improves a consumer's ability to challenge an adverse decision, a prediction future work could measure directly.
  • The discovery/justification split implies a division of labour in which technical explainability remains valuable for developers, auditors, and supervisors, while a separate justification layer becomes the only legally relevant interface with consumers.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper argues that technical XAI explanations are insufficient in business-to-consumer contexts such as credit scoring, and that a broader interpretation of the right to explanation as a legal justification ('justifiable AI') is the only way to provide consumers with actionable knowledge for contestation. The paper reviews relevant EU law, identifying art. 18(8) CCD as the central provision after the art. 86(3) AIA derogation, and draws on distinctions from legal theory between discovery and justification. It concludes that technical explanations alone constitute 'regulatory fiction' and proposes that justifiability, possibly generated by LLMs with expert rules, should be the standard.

Significance. The paper addresses a timely and important question: whether the EU's right to explanation is satisfied by technical XAI or requires a context-aware, procedurally grounded justification. It correctly identifies the key legal provisions (art. 86 AIA and its derogation, art. 18(8) CCD) and the relevant case law, and it offers a useful conceptual distinction between explanation and justification. The credit-scoring example is well chosen and keeps the argument focused. The paper does not provide machine-checked proofs or a new empirical dataset, but it advances a clear, falsifiable normative thesis and acknowledges the need for future empirical work on LLM-generated justifications.

major comments (4)
  1. [Section IV and Section V] The central claim that justifiable AI is 'the only path to genuine transparency' (Section V) is not supported by the evidence presented. The paper relies on studies involving lawyers and tax-domain users (footnotes 57 and 58) and on anecdotal experience, but it does not provide a head-to-head comparison between legal-format justifications and plain-language technical explanations with consumers deciding about credit. Because the paper explicitly concedes that justifications can be produced by LLMs fed with 'information obtained from low-level explanation-generation systems' (Section V), the operative difference appears to be presentation format, not the underlying content. Without a controlled consumer study, the superiority claim is a hypothesis, not an established conclusion.
  2. [Section III] The paper asserts that the purpose of the right to explanation is to provide actionable knowledge for contestation, but this normative premise is not defended against competing interpretations. The legal texts (GDPR, CCD, AIA) also support transparency, auditability, and institutional accountability as purposes, and the paper does not engage with authors who favor these views. This premise is load-bearing: if the right is primarily about auditability, technical XAI may be sufficient, and the argument for justifiability weakens. The paper should explicitly address and rebut alternative readings, beyond labeling some interpretations 'creative'.
  3. [Section II and Section IV] There is an internal tension about whether technical XAI can ever provide actionable knowledge. Section II describes 'actionable recourse explainers' that tell the explainee what action to take and notes that explanations can provide 'recourse (actionable knowledge)'. Section IV, however, states that technical explanations are 'alien' and that even source code or weights would 'still be insufficient to present end user with an actionable knowledge'. The paper should clarify whether the claim is that technical XAI is insufficient in practice due to design and implementation, or that it is impossible in principle. The current wording overstates the case and makes the 'regulatory fiction' rhetoric vulnerable to counterexamples.
  4. [Section I and Section V] The paper begins by saying that justifiability is 'hypothesized' as a superior standard (Section I) but concludes with the unqualified statement that it 'is the only path' (Section V). This escalation is not justified by the arguments in between. The paper should either soften the conclusion to a conditional claim or provide sufficient evidence for the strong claim. As written, the mismatch between the hypothesis language and the final categorical assertion weakens the paper's credibility.
minor comments (5)
  1. [Abstract] The phrase 'safeguard the creditors rights' appears to be a typo for 'consumers' rights' (or 'debtors' rights'), which would align with the paper's argument about consumer protection.
  2. [Section I] 'introductory remaks' should be 'introductory remarks'.
  3. [Section III] The text contains a typo 'GPPR' where 'GDPR' is intended.
  4. [Section IV] The phrase 'a capite ad calcem' is not translated; a brief gloss (e.g., 'from head to heel') would help readers unfamiliar with Latin.
  5. [References] The Busuioc, Curtin, and Almada article appears twice in the reference list under identical titles; one entry should be removed.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the paper's normative argument rests on independent legal analysis; author's prior empirical studies are supporting, externally falsifiable evidence, not a fitted input.

full rationale

This is a legal-normative paper with no mathematical derivation, no fitted parameters, and no empirical prediction that is fed back as an input. The central claim, that technical XAI explanations fail to give consumers actionable knowledge and that a broad, justification-based reading of the right to explanation is needed, is argued from the text of art. 18(8) CCD, the AIA, the GDPR, and case law, with the discovery/justification distinction used as an analytical frame rather than as a conclusion-generating mechanism. The empirical premise that XAI outputs are alien to laypeople is supported by several studies, including the author's prior work (Górski & Ramakrishna 2021; Górski et al. 2024). This is self-citation, but it is not circular: those studies are externally falsifiable empirical findings, the claim is testable through consumer studies, and the paper also cites independent work (e.g., Dieber & Kirrane; Riveiro & Thill). The conclusion that justifiability is 'the only path' is stronger than the evidence provided, and the LLM-based justification layer is admittedly future work, but these are evidentiary or overclaim concerns rather than circularity. No equation, definition, or fitted input in the paper reduces the conclusion to its own premises.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The paper introduces no numerical parameters and no testable entities. Its central claim rests on four domain assumptions: the unintelligibility of XAI to laypeople, the purpose of the right to explanation, the high-risk classification of credit scoring, and the feasibility of detaching justification from model internals. The last assumption is the weakest, as it is only sketched via future LLM-based research.

assumptions (4)
  • domain assumption Technical XAI methods (LIME, SHAP, Grad-CAM, counterfactual/recourse) are designed for developers and are not comprehensible to lay users.
    The paper's core premise is that technical explanations fail to deliver actionable knowledge. This is supported by cited empirical studies (Górski & Ramakrishna 2021; Dieber & Kirrane 2020; Górski et al. 2024) and by the author's own experience, but it is a generalization about an entire research field.
  • domain assumption The proper function of the right to explanation in consumer credit is to give the consumer actionable knowledge to contest the decision.
    This normative gloss on art. 18(8) CCD and art. 86 AIA is argued, not derived. It is the basis for rejecting technical explanations as insufficient.
  • domain assumption Credit scoring is a high-risk AI application under the AIA and CCD, so fundamental rights and consumer protection principles apply.
    The paper relies on recital 56 CCD and Annex III of the AIA to place credit scoring in the high-risk category.
  • domain assumption A legal justification can in principle be detached from the model's internal logic and still satisfy the right to explanation.
    The paper's proposed 'justifiability' standard assumes that narrative justifications based on data categories and procedural context are meaningful even when the model's internal workings remain opaque. This is presented as feasible but not demonstrated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Against Explainable Artificial Intelligence In Law: Why Justifiable Ai Matters. A Credit Scoring Example." pith.science (2026). https://pith.science/paper/UIG2WFGZ

@misc{pith2026260807452,
  author       = {Pith},
  title        = {Pith review of: Against Explainable Artificial Intelligence In Law: Why Justifiable Ai Matters. A Credit Scoring Example},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UIG2WFGZ}},
  note         = {Machine review of arXiv:2608.07452}
}
read the original abstract

Artificial intelligence-based solutions offer new efficiency-increasing possibilities in many applications, including credit scoring. Yet, the increasing sophistication of machine-learning models in use raises concerns regarding many of their aspects, explainability notwithstanding. We review the relevant EU legal background and integrate this review with insights from technical sciences to interpret relevant legal provisions in the light of technological possibilities. We reject the narrow interpretations of the right to explanation and suggest the broad one, which encompasses not only a technical explanations but also a legal justification as the only one that allows to safeguard the creditors rights in an operative manner.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

17 extracted references · 13 canonical work pages

  1. [1]

    Akoh Atadoga and others, ‘The Intersection Of Ai And Quantum Computing In Financial Markets: A Critical Review’ (2024) 5 Computer Science & IT Research Journal 461 Almada M and others, ‘Towards eXplainable Artificial Intelligence (XAI) in Tax Law: The Need for a Minimum Legal Standard’ (2022) 14 World tax journal Arkoudas K, ‘ChatGPT Is No Stochastic Parr...

    work page 2024

  2. [4]

    An Overview of the Current Legal Framework (s)’

    arXiv preprint arXiv:2012.00093 Ebers M, ‘Regulating Explainable AI in the European Union. An Overview of the Current Legal Framework (s)’

    arXiv 2012

  3. [5]

    An Overview of the Current Legal Framework (s)(August 9, 2021). Liane Colonna/Stanley Greenstein (eds.), Nordic Yearbook of Law and Informatics Eurpean Union, ‘Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the Protection of Natural Persons with Regard to the Processing of Personal Data and on the Free Movement ...

    work page 2021

  4. [8]

    arXiv preprint arXiv:2302.10766 Hildebrandt M, ‘Privacy as Protection of the Incomputable Self: From Agnostic to Agonistic Machine Learning’ (2019) 20 Theoretical Inquiries in Law 83 Hurley M and Adebayo J, ‘CREDIT SCORING IN THE ERA OF BIG DATA’ (2016) 18 Big Data Jackowski M and others, ‘First Global Report on the State of Artificial Intelligence in Leg...

    work page Pith review arXiv 2019

  5. [9]

    Right to Explanation

    Jongepier F and Keymolen E, ‘Explanation and Agency: Exploring the Normative-Epistemic Landscape of the “Right to Explanation”’ (2022) 24 Ethics and Information Technology 49 Kuiper O and others, ‘Exploring Explainable AI in the Financial Sector: Perspectives of Banks and Supervisory Authorities’ in Luis A Leiva and others (eds), Artificial Intelligence a...

    work page 2022

  6. [11]

    <https://papers.ssrn.com/abstract=4996173> accessed 24 October 2024 Miller T, ‘Explanation in Artificial Intelligence: Insights from the Social Sciences’

    work page 2024

  7. [13]

    <https://christophm.github.io/interpretable-ml-book/index.html#summary> accessed 16 May 2022 Mougan C, Kanellos G and Gottron T, ‘Desiderata for Explainable AI in Statistical Production Systems of the European Central Bank’ (arXiv, 12 February

    work page 2022

  8. [14]

    Desiderata for Explainable AI in statistical production systems of the European Central Bank

    <http://arxiv.org/abs/2107.08045> accessed 10 December 2025 Prado DPD, ‘The Challenges of Algorithm Management: The Spanish Perspective’ (2024) 29 Białostockie Studia Prawnicze 131 Riveiro M and Thill S, ‘“That’s (Not) the Output I Expected!” On the Role of End User Expectations in Creating Explanations of AI Systems’ (2021) 298 Artificial Intelligence 10...

    work page Pith review arXiv 2024

Show all 17 references
  1. [15]

    <http://arxiv.org/abs/2409.20536> accessed 10 December 2025 Wachter S, ‘Limitations and Loopholes in the EU AI Act and AI Liability Directives: What This Means for the European Union, the United States, and Beyond’

    2025 arXiv

  2. [16]

    SSRN Electronic Journal <https://www.ssrn.com/abstract=4924553> accessed 14 November 2024 21 Wasserman-Rozen H and Gilad-Bachrach R, ‘LOST IN TRANSLATION: THE LIMITS OF EXPLAINABILITY IN A1I’ (2024) 42 Wehnert S, ‘Justifiable Artificial Intelligence: Engineering Large Language...

    2024

  3. [17]

    <http://arxiv.org/abs/2311.15716> accessed 27 June 2025 Wei Y and others, ‘Credit Scoring with Social Network Data’ (2016) 35 Marketing Science 234

    2016 arXiv

  4. [2018]

    arXiv:1706.07269 [cs] <http://arxiv.org/abs/1706.07269> accessed 15 December 2019 Molnar C, Interpretable Machine Learning (Lulu.com

    2019 arXiv

  5. [2020]

    bundesbank

    URL: https://www. bundesbank. de/resource/blob/598256/d7d26167bceb18ee7c0c296902e42162/mL/2020-11-policy-dp- aiml-data. pdf Busuioc M, Curtin D and Almada M, ‘Reclaiming Transparency: Contesting the Logics of Secrecy within the AI Act’ (2023) 2 European Law Open 79 ——, ‘Reclai...

    2023

  6. [2021]

    <https://doi.org/10.1145/3462757.3466145> ——, ‘Right to Explanation in LLMs: Lessons from EU AI Act and GDPR’

  7. [2022]

    <https://link.springer.com/10.1007/978-3-030-93842-0_6> accessed 19 December 2024 20 Kumar A, Sharma S and Mahdavi M, ‘Machine Learning (ML) Technologies for Digital Credit Scoring in Rural Finance: A Literature Review’ (2021) 9 Risks 192 Langer M and others, ‘What Do We Want ...

    2021 doi

  8. [2023]

    Ghassemi M, Oakden-Rayner L and Beam AL, ‘The False Hope of Current Approaches to Explainable Artificial Intelligence in Health Care’ (2021) 3 The Lancet Digital Health e745 Górski Ł and others, ‘Exploring Explainable AI in the Tax Domain’

    2021

  9. [2024]

    <https://www.medialaws.eu/the -ai-acts-right-to- explanation-a-plea-for-an-integrated-remedy/> accessed 21 December 2024 Dieber J and Kirrane S, ‘Why Model Why? Assessing the Strengths and Limitations of LIME’

    2024

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.