REVIEW 3 major objections 5 minor 28 references
Who Benefits from AI Explanations? Towards Accessible and Interpretable Systems
T0 review · 3 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash
Pith's one-line read This paper argues that the XAI evaluation literature is built around sighted users: only 1 of 79 end-user studies includes participants with visual impairment, and the dominant techniques default to visual formats.
desk verdict A useful, honestly-reported gap analysis and prototype for accessible XAI, but the headline statistic rests on a search string that may miss relevant work and the user study has three participants. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The argument runs on two machines. First, a systematic review following a PRISMA flow diagram, with search string (XAI OR "AI explanation") AND (end-user OR user) AND (evaluation OR validation) across seven databases, whose exclusion criteria leave 79 studies; this supplies the 1-in-79 and 3-in-79 counts that define the gap. Second, a four-part methodological proof of concept—categorization of AI systems, persona definition, prototype implementation, expert and user assessment—that turns the gap into a testable design: a web prototype with a machine-learning traffic-flow model and simplified and detailed LIME and SHAP explanations, made accessible through screen readers, keyboard navigation,
What would settle it
Run the same seven-database review with search terms expanded to include 'screen reader,' 'assistive technology,' 'visual impairment,' 'accessible explanation,' and 'non-visual'; if this surfaces more than a handful of end-user XAI evaluations that include blind or low-vision participants, the claimed 1-in-79 gap is overstated. Separately, a comprehension experiment with a larger sample comparing simplified and detailed screen-reader LIME and SHAP explanations would test the preliminary finding.
Extended reading notes
Core claim
The central claim is that accessibility, specifically for people with vision impairments, is missing from the empirical XAI literature and from the default design of XAI techniques. In a review of 79 end-user evaluation studies, only three mention accessibility concerns and only one includes participants who report visual impairment; 33 of the 79 evaluate SHAP and 30 evaluate LIME, whose typical outputs are visual. The paper argues that visual-only explanations block users from contesting biased AI decisions and exclude them from meaningful participation in AI governance. Based on a small co-design session, it further argues that simplified explanations outperform detailed ones for non-visua
Load-bearing premise
The review's count of only one accessibility-aware study in 79 rests on the assumption that the search string (XAI OR "AI explanation") AND (end-user OR user) AND (evaluation OR validation) captures the whole field of end-user XAI evaluation studies, so a gap in vocabulary would not masquerade as a gap in research.
Editorial extensions
If this is right
- If the review's counts hold, published claims about how well users understand SHAP, LIME, and other XAI techniques apply to sighted users only; generalizing them to the roughly 2.2 billion people with near or distance vision impairment is unsupported.
- XAI technique choice would have to treat accessibility as a first-class criterion alongside model-agnosticism and fidelity, since the default outputs of the most popular methods are visual.
- Explanation design for non-visual users should offer simplified overviews first and detailed content as an option; the preliminary result says details without an accessible overview hurt comprehension.
- Multimodal presentation—text, audio, simplified point form, linear charts over tables—becomes a requirement for equitable access, not an enhancement.
- The four-part proof of concept offers a reusable template for evaluating any AI system's explanations with disabled users before deployment.
Reading between the lines
- A testable extension: re-run a standard task-based XAI evaluation, such as decision accuracy with SHAP versus LIME, using screen-reader output and compare outcomes to published sighted results; if accuracy gaps appear, existing XAI metrics may be measuring visual literacy as much as understanding.
- The preliminary finding implies that adding alt-text or image descriptions to existing charts may not be enough; non-visual comprehension may require a fundamentally different representation, such as sonified feature weights or narrative text.
- If the review's gap is real, it also bears on transparency regulation: an explanation that cannot be perceived is arguably no explanation at all, so accessibility standards may need to be read into AI transparency duties.
- Because the review used one narrow search string, a broader replication using terms like 'screen reader,' 'assistive technology,' and 'accessible explanation' would show whether the gap is in the literature or partly in the search vocabulary.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper addresses accessibility in explainable AI (XAI), focusing on users with vision impairments. It reports a PRISMA-style literature review of 79 studies (2019–2024) that evaluate XAI techniques with end users, concluding that only one included study involves participants with visual impairment, three mention accessibility concerns, and most explanations rely on inherently visual formats. The authors then present a four-part methodological proof of concept: a categorization of AI systems, a persona (Caroline) created with CNIB input, a functional prototype implementing simplified and detailed LIME/SHAP explanations with screen-reader support, and an evaluation with accessibility experts and users with lived experience of sight loss. Based on three active participants, the user evaluation suggests that simplified explanations are more comprehensible than detailed ones for non-visual users and that multimodal presentation is needed. The authors explicitly label the user study preliminary.
Significance. If the prevalence claim is robust, the finding that only 1 of 79 XAI evaluation studies includes participants with visual impairment is important and actionable for the XAI and HCI communities. The proof of concept is concrete, involves collaboration with accessibility experts from CNIB, follows ARIA practices, and was tested with several screen readers; this provides a useful template for future inclusive XAI design. The paper is honest about the small sample and preliminary nature of the user results. Its main weakness is that the literature-review methodology and the modality-based conclusions are not reported in sufficient detail to fully support the central prevalence claim.
major comments (3)
- [§2, search string and PRISMA flow] The single search string (XAI OR “AI explanation”) AND (end-user OR user) AND (evaluation OR validation) may systematically miss accessible-XAI studies that use different vocabulary, such as “screen reader”, “non-visual”, “blind users”, “assistive technology”, “low vision”, or “interpretable machine learning” combined with “user study” or “co-design”. Because the headline statistic (79 studies; 1 with visual-impairment participants) is the main evidence for the claimed gap, this is load-bearing. The paper does not report per-database search strings, truncation or date filters, inclusion/exclusion decision details, or screening reliability. The authors should expand and validate the search, report a protocol, and ideally add a citation/screening check against known accessible-XAI work; otherwise the 1/79 figure may overstate the field’s neglect.
- [§2 and Table S1] The claim that “most explanations rely on inherently visual formats” is not supported by systematic modality coding. Table S1 records XAI techniques, evaluation methods, metrics, and evaluators, but it does not code whether each study’s explanations were visual-only, textual, audio, haptic, or multimodal. Inferring visual dependence from technique names (SHAP, LIME, Grad-CAM) conflates the default implementation with the actual presentation used in the study; LIME and SHAP can be rendered as text or speech. A separate analysis or an additional column in Table S1 that operationalizes “inherently visual” is needed before this general conclusion is stated.
- [§3.4, user evaluation] The secondary claim—simplified explanations are more comprehensible for non-visual users than detailed ones—rests on three active participants and qualitative agreement, with no coding scheme, inter-rater reliability, or quantitative comparison. The authors correctly call this preliminary in the Conclusion, but the abstract presents it as a finding. Either report the evidence with the caveat directly in the abstract, or frame the user component as an illustrative co-design session rather than an empirical evaluation result. This does not invalidate the proof of concept, but it must not be read as a measured comparison of explanation formats.
minor comments (5)
- [§3.1] The “randomly selected combination” of AI categories is not reproducible. Specify the random selection procedure or state explicitly that it was an arbitrary illustrative choice.
- [§3.3, Algorithm 1] The pseudocode contains informal lines such as “Import lib imports”. Clean up the pseudocode and use consistent notation for functions and data structures.
- [Table S1] The row for Gunning and Aha [2019] lacks a concrete evaluation method and metrics; use “N/A” or clarify what was extracted. Also, some evaluator counts are missing (“quantity not provided”); mark these consistently.
- [§3.4] The expert evaluation with six CNIB experts is described only narratively. While useful, a short summary of how expert feedback was recorded and aggregated would improve transparency.
- [References and footnotes] Some URLs appear as footnotes and some as reference entries; format consistently according to the venue style.
Circularity Check
No significant circularity; the literature-review gap claim is independently supported by the paper's own PRISMA review, and the preliminary user finding is directly reported.
full rationale
The paper does not contain a mathematical derivation in which an output equals an input by construction. The central claim—that XAI evaluation studies rarely include disabled users and rely on visual formats—is an empirical summary of the 79 papers the authors screened (Section 2, Figure 1, Table S1). The search-string limitation (e.g., missing studies that use accessibility vocabulary) is a validity/correctness concern, not circularity. The only self-citation is to the authors' prior survey [Nwokoye et al., 2024], used to motivate the gap, but the gap is re-established by the new independent review, so the self-citation is not load-bearing. The preliminary finding that simplified explanations are more comprehensible is explicitly based on three participants' feedback and is labeled preliminary in Sections 3.4 and 4; it is not a fitted parameter renamed as a prediction. No uniqueness theorem, ansatz, or renaming of a known result is invoked. Therefore no circular step is exhibited; score reflects the minor non-load-bearing self-citation only.
Assumptions & free parameters
assumptions (4)
- domain assumption The search string (XAI OR 'AI explanation') AND (end-user OR user) AND (evaluation OR validation) captures the population of XAI evaluation studies.
- domain assumption A paper that does not mention accessibility or visual impairment did not consider disabled users.
- domain assumption Self-reported comprehension in a one-hour online co-design session is a valid measure of explanation comprehension.
- domain assumption The single persona 'Caroline' adequately represents a lived-experience user for the scenario design.
invented entities (1)
-
Persona 'Caroline' (traffic manager, totally blind since birth)
Cite this review
Pith. "Pith review of Who Benefits from AI Explanations? Towards Accessible and Interpretable Systems." pith.science (2026). https://pith.science/paper/QHWU6TWF
@misc{pith2026250810806,
author = {Pith},
title = {Pith review of: Who Benefits from AI Explanations? Towards Accessible and Interpretable Systems},
year = {2026},
howpublished = {\url{https://pith.science/paper/QHWU6TWF}},
note = {Machine review of arXiv:2508.10806}
}
read the original abstract
As AI systems are increasingly deployed to support decision-making in critical domains, explainability has become a means to enhance the understandability of these outputs and enable users to make more informed and conscious choices. However, despite growing interest in the usability of eXplainable AI (XAI), the accessibility of these methods, particularly for users with vision impairments, remains underexplored. This paper investigates accessibility gaps in XAI through a two-pronged approach. First, a literature review of 79 studies reveals that evaluations of XAI techniques rarely include disabled users, with most explanations relying on inherently visual formats. Second, we present a four-part methodological proof of concept that operationalizes inclusive XAI design: (1) categorization of AI systems, (2) persona definition and contextualization, (3) prototype design and implementation, and (4) expert and user assessment of XAI techniques for accessibility. Preliminary findings suggest that simplified explanations are more comprehensible for non-visual users than detailed ones, and that multimodal presentation is required for more equitable interpretability.
Figures
Reference graph
Works this paper leans on
-
[1]
Comparing user perception of explanations developed with XAI methods
Jonathan Aechtner, Lena Cabrera, Dennis Katwal, Pierre Onghena, Diego Penroz Valenzuela, and Anna Wilbik. Comparing user perception of explanations developed with XAI methods. In 2022 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), pages 1–7. IEEE,
work page 2022
-
[3]
Explainable human-robot interaction for imitation learning in aug- mented reality
Anna Belardinelli, Chao Wang, and Michael Gienger. Explainable human-robot interaction for imitation learning in aug- mented reality. In Cristina Piazza, Patricia Capsi-Morales, Luis Figueredo, Manuel Keppler, and Hinrich Sch¨ utze, editors, Human-Friendly Robotics 2023 , volume 29, pages 94–109. Springer Nature Switzerland,
work page 2023
-
[7]
Shedding light on the black box: Explainable AI for predicting household appliance failures
Taha Falatouri, Mehran Nasseri, Patrick Brandtner, and Farzaneh Darbanian. Shedding light on the black box: Explainable AI for predicting household appliance failures. In Helmut Degen, Stavroula Ntoa, and Abbas Moallem, editors, HCI International 2023 – Late Breaking Papers , volume 14059, pages 69–83. Springer Nature Switzerland,
work page 2023
-
[9]
IRTEX: Image retrieval with textual explanations
Tarun Gupta, Libin Kutty, Ritu Gahir, Nnamdi Ukwu, Sayantan Polley, and Marcus Thiel. IRTEX: Image retrieval with textual explanations. In 2021 IEEE 2nd International Conference on Human-Machine Systems (ICHMS) , pages 1–4. IEEE,
work page 2021
-
[10]
An interactive XAI interface with application in healthcare for non-experts
Jingyu Hu, Yizhu Liang, Weiyu Zhao, Kevin McAreavey, and Weiru Liu. An interactive XAI interface with application in healthcare for non-experts. In Luca Longo, editor, Explainable Artificial Intelligence, volume 1901, pages 649–670. Springer Nature Switzerland,
work page 1901
-
[11]
Lujain Ibrahim, Mohammad M Ghassemi, and Tuka Alhanai
Series Title: Communications in Computer and Information Science. Lujain Ibrahim, Mohammad M Ghassemi, and Tuka Alhanai. Do explanations improve the quality of AI-assisted human deci- sions? An algorithm-in-the-loop analysis of factual & counterfactual explanations. In Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent Sy...
work page 2023
-
[12]
How can i choose an explainer?: An application-grounded evaluation of post-hoc explanations
S´ ergio Jesus, Catarina Bel´ em, Vladimir Balayan, Jo˜ ao Bento, Pedro Saleiro, Pedro Bizarro, and Jo˜ ao Gama. How can i choose an explainer?: An application-grounded evaluation of post-hoc explanations. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, pages 805–815. ACM,
work page 2021
-
[14]
Sunnie S. Y. Kim, Nicole Meister, Vikram V. Ramaswamy, Ruth Fong, and Olga Russakovsky. HIVE: Evaluating the human interpretability of visual explanations. In Shai Avidan, Gabriel Brostow, Moustapha Ciss´ e, Giovanni Maria Farinella, and Tal Hassner, editors, Computer Vision – ECCV 2022 , volume 13672, pages 280–298. Springer Nature Switzerland,
work page 2022
Show all 28 references
-
[15]
Pattern recognition using artificial neural network: a review
Tai-hoon Kim. Pattern recognition using artificial neural network: a review. In Information Security and Assurance: 4th International Conference, ISA 2010, Miyazaki, Japan, June 23-25,
2010
-
[16]
Carson K
Series Title: Communications in Computer and Information Science. Carson K. Leung, Adam G.M. Pazdor, and Joglas Souza. Explainable artificial intelligence for data science on customer churn. In 2021 IEEE 8th International Conference on Data Science and Advanced Analytics (DSAA...
2021
-
[17]
Explainable AI for non-experts: Energy tariff forecasting
Hongnan Ma, Kevin McAreavey, Ryan McConville, and Weiru Liu. Explainable AI for non-experts: Energy tariff forecasting. In 2022 27th International Conference on Automation and Computing (ICAC) , pages 1–6. IEEE,
2022
-
[18]
Interfaces for explanations in human-AI interaction: Proposing a design evaluation approach
Henrik Mucha, Sebastian Robert, Ruediger Breitschwerdt, and Michael Fellmann. Interfaces for explanations in human-AI interaction: Proposing a design evaluation approach. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems , pages 1–6. ACM,
2021
-
[19]
On the impact of explanations on under- standing of algorithmic decision-making
Timoth´ ee Schmude, Laura Koesten, Torsten M¨ oller, and Sebastian Tschiatschek. On the impact of explanations on under- standing of algorithmic decision-making. In 2023 ACM Conference on Fairness, Accountability, and Transparency , pages 959–970. ACM,
2023
-
[20]
Ritu Singh
Series Title: Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering. Ritu Singh. Understanding image classification tasks through layerwise relevance propogation. In 2022 IEEE 18th Interna- tional Conference on Intelligent ...
2022
-
[21]
Evaluation of human-understandability of global model explanations using decision tree
Adarsa Sivaprasad, Ehud Reiter, Nava Tintarev, and Nir Oren. Evaluation of human-understandability of global model explanations using decision tree. In S/suppress lawomir Nowaczyk, Przemys/suppress law Biecek, Neo Christopher Chung, Mauro Vallati, Pawe/suppress l Skruch, Joann...
2023
-
[22]
Trust in AI-assisted Decision Making: Perspectives from Those Behind the System and Those for Whom the Decision is Made
Oleksandra Vereschak, Fatemeh Alizadeh, Gilles Bailly, and Baptiste Caramiaux. Trust in AI-assisted Decision Making: Perspectives from Those Behind the System and Those for Whom the Decision is Made. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Syste...
2024
-
[23]
Giulia Vilone and Luca Longo
Association for Computing Machinery. Giulia Vilone and Luca Longo. Development of a human-centred psychometric test for the evaluation of explanations produced by XAI methods. In Luca Longo, editor, Explainable Artificial Intelligence , volume 1903, pages 205–232. Springer Nat...
1903
-
[26]
Veldhuis, Simone Ari ¨ens, Rolf J.F
[Veldhuis et al., 2022] Marthe S. Veldhuis, Simone Ari ¨ens, Rolf J.F. Ypma, Thomas Abeel, and Corina C.G. Ben- schop. Explainable artificial intelligence in forensics: Re- alistic explanations for number of contributor predictions of DNA profiles. Forensic Science Internation...
2022
-
[27]
Web Content Accessi- bility Guidelines (WCAG) 2.1
[W3C Web Accessibility Initiative (W AI), 2024] W3C Web Accessibility Initiative (W AI). Web Content Accessi- bility Guidelines (WCAG) 2.1. https://www.w3.org/TR/ WCAG21/,
2024
-
[28]
Disability, 2023
[World Health Organization, 2023b] World Health Organi- zation. Disability, 2023
2023
-
[2002]
Dialogue explanation with reasoning for AI
Yifan Xu. Dialogue explanation with reasoning for AI. In Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society, pages 918–918. ACM,
2022
-
[2011]
Making SHAP rap: Bridging local and global insights through interaction and narratives
Michael Chromik. Making SHAP rap: Bridging local and global insights through interaction and narratives. In Carmelo Ardito, Rosa Lanzilotti, Alessio Malizia, Helen Petrie, Antonio Piccinno, Giuseppe Desolda, and Kori Inkpen, editors, Human-Computer Interaction – INTERACT 2021 ...
2021
-
[2019]
MiMICRI: Towards domain-centered counterfac- tual explanations of cardiovascular image classification models
Grace Guo, Lifu Deng, Animesh Tandon, Alex Endert, and Bum Chul Kwon. MiMICRI: Towards domain-centered counterfac- tual explanations of cardiovascular image classification models. In The 2024 ACM Conference on Fairness, Accountability, and Transparency, pages 1861–1874. ACM,
2024
-
[2020]
Features of Explainability: How users understand counterfactual and causal explanations for categorical and continuous features in XAI
Greta Warren, Mark T Keane, and Ruth MJ Byrne. Features of Explainability: How users understand counterfactual and causal explanations for categorical and continuous features in XAI. arXiv preprint arXiv:2204.10152 ,
-
[2021]
A review and analysis of case-based reasoning research
Chen Jian, Teng Zhe, and Liu Zhenxing. A review and analysis of case-based reasoning research. In 2015 International Conference on Intelligent Transportation, Big Data and Smart City , pages 51–55,
2015
-
[2022]
Explaining human activities instances using deep learning classifiers
Luca Arrotta, Gabriele Civitarese, Michele Fiori, and Claudio Bettini. Explaining human activities instances using deep learning classifiers. In 2022 IEEE 9th International Conference on Data Science and Advanced Analytics (DSAA) , pages 1–10. IEEE,
2022
-
[2023]
Predictive analysis using machine learning: Re- view of trends and methods
Patrick Loola Bokonda, Khadija Ouazzani-Touhami, and Nissrine Souissi. Predictive analysis using machine learning: Re- view of trends and methods. In 2020 International Symposium on Advanced Electrical and Communication Technologies (ISAECT), pages 1–6,
2020
-
[2024]
Corentin Boidot, Olivier Augereau, Pierre De Loor, and Riwal Lefort
Series Title: Springer Proceedings in Advanced Robotics. Corentin Boidot, Olivier Augereau, Pierre De Loor, and Riwal Lefort. Benefits of using multiple post-hoc explanations for machine learning. In 2023 International Conference on Machine Learning and Applications (ICMLA) , ...
2023
Reviewed August 5, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.