Pith. sign in

REVIEW 3 major objections 5 minor 28 references

Who Benefits from AI Explanations? Towards Accessible and Interpretable Systems

T0 review · 3 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Pith's one-line read This paper argues that the XAI evaluation literature is built around sighted users: only 1 of 79 end-user studies includes participants with visual impairment, and the dominant techniques default to visual formats.

desk verdict A useful, honestly-reported gap analysis and prototype for accessible XAI, but the headline statistic rests on a search string that may miss relevant work and the user study has three participants. read the letter →

arxiv 2508.10806 v1 pith:QHWU6TWF submitted 2025-08-14 cs.AI

classification cs.AI
keywords explainableAIaccessibilityvisualimpairmentnon-visualexplanationsSHAPLIMEuserevaluationinclusivedesign
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that explainable AI has an accessibility gap: across 79 end-user evaluation studies published from 2019 to 2024, only one includes participants with any visual impairment, and only three mention accessibility at all. The dominant techniques—SHAP, LIME, Grad-CAM, decision trees—are evaluated and displayed in visual formats such as bar charts, heatmaps, and graphs, so the field's evidence about 'understandable AI' is built almost entirely on sighted users. To show the gap can be closed, the authors build a screen-reader-friendly traffic-prediction prototype offering simplified and detailed LIME and SHAP explanations, then assess it with accessibility experts and three users with lived experience of sight loss. Their preliminary finding is that simplified explanations are understood better than detailed ones by non-visual users, and that multimodal presentation is needed for equitable interpretability.

What carries the argument

The argument runs on two machines. First, a systematic review following a PRISMA flow diagram, with search string (XAI OR "AI explanation") AND (end-user OR user) AND (evaluation OR validation) across seven databases, whose exclusion criteria leave 79 studies; this supplies the 1-in-79 and 3-in-79 counts that define the gap. Second, a four-part methodological proof of concept—categorization of AI systems, persona definition, prototype implementation, expert and user assessment—that turns the gap into a testable design: a web prototype with a machine-learning traffic-flow model and simplified and detailed LIME and SHAP explanations, made accessible through screen readers, keyboard navigation,

What would settle it

Run the same seven-database review with search terms expanded to include 'screen reader,' 'assistive technology,' 'visual impairment,' 'accessible explanation,' and 'non-visual'; if this surfaces more than a handful of end-user XAI evaluations that include blind or low-vision participants, the claimed 1-in-79 gap is overstated. Separately, a comprehension experiment with a larger sample comparing simplified and detailed screen-reader LIME and SHAP explanations would test the preliminary finding.

Watch

Extended reading notes

Core claim

The central claim is that accessibility, specifically for people with vision impairments, is missing from the empirical XAI literature and from the default design of XAI techniques. In a review of 79 end-user evaluation studies, only three mention accessibility concerns and only one includes participants who report visual impairment; 33 of the 79 evaluate SHAP and 30 evaluate LIME, whose typical outputs are visual. The paper argues that visual-only explanations block users from contesting biased AI decisions and exclude them from meaningful participation in AI governance. Based on a small co-design session, it further argues that simplified explanations outperform detailed ones for non-visua

Load-bearing premise

The review's count of only one accessibility-aware study in 79 rests on the assumption that the search string (XAI OR "AI explanation") AND (end-user OR user) AND (evaluation OR validation) captures the whole field of end-user XAI evaluation studies, so a gap in vocabulary would not masquerade as a gap in research.

Editorial extensions

If this is right

  • If the review's counts hold, published claims about how well users understand SHAP, LIME, and other XAI techniques apply to sighted users only; generalizing them to the roughly 2.2 billion people with near or distance vision impairment is unsupported.
  • XAI technique choice would have to treat accessibility as a first-class criterion alongside model-agnosticism and fidelity, since the default outputs of the most popular methods are visual.
  • Explanation design for non-visual users should offer simplified overviews first and detailed content as an option; the preliminary result says details without an accessible overview hurt comprehension.
  • Multimodal presentation—text, audio, simplified point form, linear charts over tables—becomes a requirement for equitable access, not an enhancement.
  • The four-part proof of concept offers a reusable template for evaluating any AI system's explanations with disabled users before deployment.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension: re-run a standard task-based XAI evaluation, such as decision accuracy with SHAP versus LIME, using screen-reader output and compare outcomes to published sighted results; if accuracy gaps appear, existing XAI metrics may be measuring visual literacy as much as understanding.
  • The preliminary finding implies that adding alt-text or image descriptions to existing charts may not be enough; non-visual comprehension may require a fundamentally different representation, such as sonified feature weights or narrative text.
  • If the review's gap is real, it also bears on transparency regulation: an explanation that cannot be perceived is arguably no explanation at all, so accessibility standards may need to be read into AI transparency duties.
  • Because the review used one narrow search string, a broader replication using terms like 'screen reader,' 'assistive technology,' and 'accessible explanation' would show whether the gap is in the literature or partly in the search vocabulary.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper addresses accessibility in explainable AI (XAI), focusing on users with vision impairments. It reports a PRISMA-style literature review of 79 studies (2019–2024) that evaluate XAI techniques with end users, concluding that only one included study involves participants with visual impairment, three mention accessibility concerns, and most explanations rely on inherently visual formats. The authors then present a four-part methodological proof of concept: a categorization of AI systems, a persona (Caroline) created with CNIB input, a functional prototype implementing simplified and detailed LIME/SHAP explanations with screen-reader support, and an evaluation with accessibility experts and users with lived experience of sight loss. Based on three active participants, the user evaluation suggests that simplified explanations are more comprehensible than detailed ones for non-visual users and that multimodal presentation is needed. The authors explicitly label the user study preliminary.

Significance. If the prevalence claim is robust, the finding that only 1 of 79 XAI evaluation studies includes participants with visual impairment is important and actionable for the XAI and HCI communities. The proof of concept is concrete, involves collaboration with accessibility experts from CNIB, follows ARIA practices, and was tested with several screen readers; this provides a useful template for future inclusive XAI design. The paper is honest about the small sample and preliminary nature of the user results. Its main weakness is that the literature-review methodology and the modality-based conclusions are not reported in sufficient detail to fully support the central prevalence claim.

major comments (3)
  1. [§2, search string and PRISMA flow] The single search string (XAI OR “AI explanation”) AND (end-user OR user) AND (evaluation OR validation) may systematically miss accessible-XAI studies that use different vocabulary, such as “screen reader”, “non-visual”, “blind users”, “assistive technology”, “low vision”, or “interpretable machine learning” combined with “user study” or “co-design”. Because the headline statistic (79 studies; 1 with visual-impairment participants) is the main evidence for the claimed gap, this is load-bearing. The paper does not report per-database search strings, truncation or date filters, inclusion/exclusion decision details, or screening reliability. The authors should expand and validate the search, report a protocol, and ideally add a citation/screening check against known accessible-XAI work; otherwise the 1/79 figure may overstate the field’s neglect.
  2. [§2 and Table S1] The claim that “most explanations rely on inherently visual formats” is not supported by systematic modality coding. Table S1 records XAI techniques, evaluation methods, metrics, and evaluators, but it does not code whether each study’s explanations were visual-only, textual, audio, haptic, or multimodal. Inferring visual dependence from technique names (SHAP, LIME, Grad-CAM) conflates the default implementation with the actual presentation used in the study; LIME and SHAP can be rendered as text or speech. A separate analysis or an additional column in Table S1 that operationalizes “inherently visual” is needed before this general conclusion is stated.
  3. [§3.4, user evaluation] The secondary claim—simplified explanations are more comprehensible for non-visual users than detailed ones—rests on three active participants and qualitative agreement, with no coding scheme, inter-rater reliability, or quantitative comparison. The authors correctly call this preliminary in the Conclusion, but the abstract presents it as a finding. Either report the evidence with the caveat directly in the abstract, or frame the user component as an illustrative co-design session rather than an empirical evaluation result. This does not invalidate the proof of concept, but it must not be read as a measured comparison of explanation formats.
minor comments (5)
  1. [§3.1] The “randomly selected combination” of AI categories is not reproducible. Specify the random selection procedure or state explicitly that it was an arbitrary illustrative choice.
  2. [§3.3, Algorithm 1] The pseudocode contains informal lines such as “Import lib imports”. Clean up the pseudocode and use consistent notation for functions and data structures.
  3. [Table S1] The row for Gunning and Aha [2019] lacks a concrete evaluation method and metrics; use “N/A” or clarify what was extracted. Also, some evaluator counts are missing (“quantity not provided”); mark these consistently.
  4. [§3.4] The expert evaluation with six CNIB experts is described only narratively. While useful, a short summary of how expert feedback was recorded and aggregated would improve transparency.
  5. [References and footnotes] Some URLs appear as footnotes and some as reference entries; format consistently according to the venue style.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity; the literature-review gap claim is independently supported by the paper's own PRISMA review, and the preliminary user finding is directly reported.

full rationale

The paper does not contain a mathematical derivation in which an output equals an input by construction. The central claim—that XAI evaluation studies rarely include disabled users and rely on visual formats—is an empirical summary of the 79 papers the authors screened (Section 2, Figure 1, Table S1). The search-string limitation (e.g., missing studies that use accessibility vocabulary) is a validity/correctness concern, not circularity. The only self-citation is to the authors' prior survey [Nwokoye et al., 2024], used to motivate the gap, but the gap is re-established by the new independent review, so the self-citation is not load-bearing. The preliminary finding that simplified explanations are more comprehensible is explicitly based on three participants' feedback and is labeled preliminary in Sections 3.4 and 4; it is not a fitted parameter renamed as a prediction. No uniqueness theorem, ansatz, or renaming of a known result is invoked. Therefore no circular step is exhibited; score reflects the minor non-load-bearing self-citation only.

Assumptions & free parameters 0 free parameters · 4 assumptions · 1 invented entities

The paper is empirical and design-oriented, so the 'free parameters' ledger is empty. The central claims rest on dataset-selection assumptions (search string coverage, absence-as-exclusion), on treating self-report as comprehension, and on a deliberately constructed persona. These are reasonable domain assumptions for a proof of concept but they are not independently verified.

assumptions (4)
  • domain assumption The search string (XAI OR 'AI explanation') AND (end-user OR user) AND (evaluation OR validation) captures the population of XAI evaluation studies.
    Section 2. If this string misses accessible-XAI studies phrased differently, the claimed gap is overstated.
  • domain assumption A paper that does not mention accessibility or visual impairment did not consider disabled users.
    Section 2. Absence of mention is treated as absence of inclusion; this is an interpretation, not a verified fact.
  • domain assumption Self-reported comprehension in a one-hour online co-design session is a valid measure of explanation comprehension.
    Section 3.4. Participants were asked what information they extracted and which method they preferred; no objective comprehension test was administered.
  • domain assumption The single persona 'Caroline' adequately represents a lived-experience user for the scenario design.
    Section 3.2. The authors explicitly state this is an illustrative persona, not a representation of all sight-loss profiles.
invented entities (1)
  • Persona 'Caroline' (traffic manager, totally blind since birth)
    purpose: Grounds the case study and evaluation context for non-visual XAI interaction.
    An illustrative design construct created with CNIB experts; it is not a measured entity and is acknowledged as not representative of all sight-loss users (Section 3.2).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Who Benefits from AI Explanations? Towards Accessible and Interpretable Systems." pith.science (2026). https://pith.science/paper/QHWU6TWF

@misc{pith2026250810806,
  author       = {Pith},
  title        = {Pith review of: Who Benefits from AI Explanations? Towards Accessible and Interpretable Systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QHWU6TWF}},
  note         = {Machine review of arXiv:2508.10806}
}
read the original abstract

As AI systems are increasingly deployed to support decision-making in critical domains, explainability has become a means to enhance the understandability of these outputs and enable users to make more informed and conscious choices. However, despite growing interest in the usability of eXplainable AI (XAI), the accessibility of these methods, particularly for users with vision impairments, remains underexplored. This paper investigates accessibility gaps in XAI through a two-pronged approach. First, a literature review of 79 studies reveals that evaluations of XAI techniques rarely include disabled users, with most explanations relying on inherently visual formats. Second, we present a four-part methodological proof of concept that operationalizes inclusive XAI design: (1) categorization of AI systems, (2) persona definition and contextualization, (3) prototype design and implementation, and (4) expert and user assessment of XAI techniques for accessibility. Preliminary findings suggest that simplified explanations are more comprehensible for non-visual users than detailed ones, and that multimodal presentation is required for more equitable interpretability.

Figures

Figures reproduced from arXiv: 2508.10806 by the authors.

Figure 1
Figure 1. PRISMA Flow Diagram to reduce barriers and ensure that individuals with sight loss and other disabilities can access and participate in the design and development of AI-based systems, particularly concern￾ing their ability to understand the decisions and recommen￾dations provided by these applications. However, the sight loss community has been largely overlooked when it comes to access to AI explanations [Nwokoye e… view at source ↗
Figure 2
Figure 2. Prototype created for the urban traffic scenario [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 5
Figure 5. Default explanation generated from SHAP - Detailed Ex [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figures from the paper (1 more)
Figure 3
Figure 3. Figure 3: XAI techniques available in the prototype. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

28 extracted references · 27 canonical work pages

  1. [1]

    Comparing user perception of explanations developed with XAI methods

    Jonathan Aechtner, Lena Cabrera, Dennis Katwal, Pierre Onghena, Diego Penroz Valenzuela, and Anna Wilbik. Comparing user perception of explanations developed with XAI methods. In 2022 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), pages 1–7. IEEE,

  2. [3]

    Explainable human-robot interaction for imitation learning in aug- mented reality

    Anna Belardinelli, Chao Wang, and Michael Gienger. Explainable human-robot interaction for imitation learning in aug- mented reality. In Cristina Piazza, Patricia Capsi-Morales, Luis Figueredo, Manuel Keppler, and Hinrich Sch¨ utze, editors, Human-Friendly Robotics 2023 , volume 29, pages 94–109. Springer Nature Switzerland,

  3. [7]

    Shedding light on the black box: Explainable AI for predicting household appliance failures

    Taha Falatouri, Mehran Nasseri, Patrick Brandtner, and Farzaneh Darbanian. Shedding light on the black box: Explainable AI for predicting household appliance failures. In Helmut Degen, Stavroula Ntoa, and Abbas Moallem, editors, HCI International 2023 – Late Breaking Papers , volume 14059, pages 69–83. Springer Nature Switzerland,

  4. [9]

    IRTEX: Image retrieval with textual explanations

    Tarun Gupta, Libin Kutty, Ritu Gahir, Nnamdi Ukwu, Sayantan Polley, and Marcus Thiel. IRTEX: Image retrieval with textual explanations. In 2021 IEEE 2nd International Conference on Human-Machine Systems (ICHMS) , pages 1–4. IEEE,

  5. [10]

    An interactive XAI interface with application in healthcare for non-experts

    Jingyu Hu, Yizhu Liang, Weiyu Zhao, Kevin McAreavey, and Weiru Liu. An interactive XAI interface with application in healthcare for non-experts. In Luca Longo, editor, Explainable Artificial Intelligence, volume 1901, pages 649–670. Springer Nature Switzerland,

  6. [11]

    Lujain Ibrahim, Mohammad M Ghassemi, and Tuka Alhanai

    Series Title: Communications in Computer and Information Science. Lujain Ibrahim, Mohammad M Ghassemi, and Tuka Alhanai. Do explanations improve the quality of AI-assisted human deci- sions? An algorithm-in-the-loop analysis of factual & counterfactual explanations. In Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent Sy...

  7. [12]

    How can i choose an explainer?: An application-grounded evaluation of post-hoc explanations

    S´ ergio Jesus, Catarina Bel´ em, Vladimir Balayan, Jo˜ ao Bento, Pedro Saleiro, Pedro Bizarro, and Jo˜ ao Gama. How can i choose an explainer?: An application-grounded evaluation of post-hoc explanations. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, pages 805–815. ACM,

  8. [14]

    Sunnie S. Y. Kim, Nicole Meister, Vikram V. Ramaswamy, Ruth Fong, and Olga Russakovsky. HIVE: Evaluating the human interpretability of visual explanations. In Shai Avidan, Gabriel Brostow, Moustapha Ciss´ e, Giovanni Maria Farinella, and Tal Hassner, editors, Computer Vision – ECCV 2022 , volume 13672, pages 280–298. Springer Nature Switzerland,

Show all 28 references
  1. [15]

    Pattern recognition using artificial neural network: a review

    Tai-hoon Kim. Pattern recognition using artificial neural network: a review. In Information Security and Assurance: 4th International Conference, ISA 2010, Miyazaki, Japan, June 23-25,

  2. [16]

    Carson K

    Series Title: Communications in Computer and Information Science. Carson K. Leung, Adam G.M. Pazdor, and Joglas Souza. Explainable artificial intelligence for data science on customer churn. In 2021 IEEE 8th International Conference on Data Science and Advanced Analytics (DSAA...

  3. [17]

    Explainable AI for non-experts: Energy tariff forecasting

    Hongnan Ma, Kevin McAreavey, Ryan McConville, and Weiru Liu. Explainable AI for non-experts: Energy tariff forecasting. In 2022 27th International Conference on Automation and Computing (ICAC) , pages 1–6. IEEE,

  4. [18]

    Interfaces for explanations in human-AI interaction: Proposing a design evaluation approach

    Henrik Mucha, Sebastian Robert, Ruediger Breitschwerdt, and Michael Fellmann. Interfaces for explanations in human-AI interaction: Proposing a design evaluation approach. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems , pages 1–6. ACM,

  5. [19]

    On the impact of explanations on under- standing of algorithmic decision-making

    Timoth´ ee Schmude, Laura Koesten, Torsten M¨ oller, and Sebastian Tschiatschek. On the impact of explanations on under- standing of algorithmic decision-making. In 2023 ACM Conference on Fairness, Accountability, and Transparency , pages 959–970. ACM,

  6. [20]

    Ritu Singh

    Series Title: Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering. Ritu Singh. Understanding image classification tasks through layerwise relevance propogation. In 2022 IEEE 18th Interna- tional Conference on Intelligent ...

  7. [21]

    Evaluation of human-understandability of global model explanations using decision tree

    Adarsa Sivaprasad, Ehud Reiter, Nava Tintarev, and Nir Oren. Evaluation of human-understandability of global model explanations using decision tree. In S/suppress lawomir Nowaczyk, Przemys/suppress law Biecek, Neo Christopher Chung, Mauro Vallati, Pawe/suppress l Skruch, Joann...

  8. [22]

    Trust in AI-assisted Decision Making: Perspectives from Those Behind the System and Those for Whom the Decision is Made

    Oleksandra Vereschak, Fatemeh Alizadeh, Gilles Bailly, and Baptiste Caramiaux. Trust in AI-assisted Decision Making: Perspectives from Those Behind the System and Those for Whom the Decision is Made. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Syste...

  9. [23]

    Giulia Vilone and Luca Longo

    Association for Computing Machinery. Giulia Vilone and Luca Longo. Development of a human-centred psychometric test for the evaluation of explanations produced by XAI methods. In Luca Longo, editor, Explainable Artificial Intelligence , volume 1903, pages 205–232. Springer Nat...

  10. [26]

    Veldhuis, Simone Ari ¨ens, Rolf J.F

    [Veldhuis et al., 2022] Marthe S. Veldhuis, Simone Ari ¨ens, Rolf J.F. Ypma, Thomas Abeel, and Corina C.G. Ben- schop. Explainable artificial intelligence in forensics: Re- alistic explanations for number of contributor predictions of DNA profiles. Forensic Science Internation...

  11. [27]

    Web Content Accessi- bility Guidelines (WCAG) 2.1

    [W3C Web Accessibility Initiative (W AI), 2024] W3C Web Accessibility Initiative (W AI). Web Content Accessi- bility Guidelines (WCAG) 2.1. https://www.w3.org/TR/ WCAG21/,

  12. [28]

    Disability, 2023

    [World Health Organization, 2023b] World Health Organi- zation. Disability, 2023

  13. [2002]

    Dialogue explanation with reasoning for AI

    Yifan Xu. Dialogue explanation with reasoning for AI. In Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society, pages 918–918. ACM,

  14. [2011]

    Making SHAP rap: Bridging local and global insights through interaction and narratives

    Michael Chromik. Making SHAP rap: Bridging local and global insights through interaction and narratives. In Carmelo Ardito, Rosa Lanzilotti, Alessio Malizia, Helen Petrie, Antonio Piccinno, Giuseppe Desolda, and Kori Inkpen, editors, Human-Computer Interaction – INTERACT 2021 ...

  15. [2019]

    MiMICRI: Towards domain-centered counterfac- tual explanations of cardiovascular image classification models

    Grace Guo, Lifu Deng, Animesh Tandon, Alex Endert, and Bum Chul Kwon. MiMICRI: Towards domain-centered counterfac- tual explanations of cardiovascular image classification models. In The 2024 ACM Conference on Fairness, Accountability, and Transparency, pages 1861–1874. ACM,

  16. [2020]

    Features of Explainability: How users understand counterfactual and causal explanations for categorical and continuous features in XAI

    Greta Warren, Mark T Keane, and Ruth MJ Byrne. Features of Explainability: How users understand counterfactual and causal explanations for categorical and continuous features in XAI. arXiv preprint arXiv:2204.10152 ,

  17. [2021]

    A review and analysis of case-based reasoning research

    Chen Jian, Teng Zhe, and Liu Zhenxing. A review and analysis of case-based reasoning research. In 2015 International Conference on Intelligent Transportation, Big Data and Smart City , pages 51–55,

  18. [2022]

    Explaining human activities instances using deep learning classifiers

    Luca Arrotta, Gabriele Civitarese, Michele Fiori, and Claudio Bettini. Explaining human activities instances using deep learning classifiers. In 2022 IEEE 9th International Conference on Data Science and Advanced Analytics (DSAA) , pages 1–10. IEEE,

  19. [2023]

    Predictive analysis using machine learning: Re- view of trends and methods

    Patrick Loola Bokonda, Khadija Ouazzani-Touhami, and Nissrine Souissi. Predictive analysis using machine learning: Re- view of trends and methods. In 2020 International Symposium on Advanced Electrical and Communication Technologies (ISAECT), pages 1–6,

  20. [2024]

    Corentin Boidot, Olivier Augereau, Pierre De Loor, and Riwal Lefort

    Series Title: Springer Proceedings in Advanced Robotics. Corentin Boidot, Olivier Augereau, Pierre De Loor, and Riwal Lefort. Benefits of using multiple post-hoc explanations for machine learning. In 2023 International Conference on Machine Learning and Applications (ICMLA) , ...

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.