REVIEW 45 cited by
Metrics for Explainable AI: Challenges and Prospects
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
The question addressed in this paper is: If we present to a user an AI system that explains how it works, how do we know whether the explanation works and the user has achieved a pragmatic understanding of the AI? In other words, how do we know that an explanainable AI system (XAI) is any good? Our focus is on the key concepts of measurement. We discuss specific methods for evaluating: (1) the goodness of explanations, (2) whether users are satisfied by explanations, (3) how well users understand the AI systems, (4) how curiosity motivates the search for explanations, (5) whether the user's trust and reliance on the AI are appropriate, and finally, (6) how the human-XAI work system performs. The recommendations we present derive from our integration of extensive research literatures and our own psychometric evaluations.
Forward citations
Cited by 45 Pith papers
-
What Should Explanations Contain? A Human-Centered Explanation Content Model for Local, Post-Hoc Explanations
A 14-code content model for local post-hoc AI explanations, derived from 325 user statements and validated by experts with high reliability scores.
-
SocialFiVis: A Visual Analytics Sandbox for LLM-Grounded Multi-Agent Simulation in Social Finance
SocialFiVis couples LLM-derived personas with a mechanism-guided simulation and a multi-view interface to support counterfactual governance analysis in SocialFi communities, evaluated through case studies and a 13-par...
-
"Trust Junk" Leads to Unjustified Support for Highly Discriminatory Predictive Models
Accurate but irrelevant 'trust junk' in AI explanations made crowdsourced users trust and agree with a deliberately discriminatory model more.
-
Supporting Calibrated Reliance in Human-AI Collaboration: Different Strategies for Different Tasks
LLM explanations create a persuasion paradox by increasing confidence without accuracy gains in visual tasks while aiding logical tasks, with uncertainty displays and selective automation outperforming explanations in...
-
Visual-Conversational Interface for Evidence-Based Explanation of Diabetes Risk Prediction
A visual-conversational diabetes risk tool grounded in scientific evidence was rated by 30 healthcare professionals as improving understanding and calibrating trust.
-
Why is My Route Different Today? An Algorithm for Explaining Route Selection
SVEs are LP-minimal sets of traffic-affected road segments that make a user's chosen route the shortest route, and the paper gives a flow-based algorithm plus experiments showing they are small.
-
Understanding Mental Models of Generative Conversational Search and The Effect of Interface Transparency
Users of generative conversational search mostly hold abstract, incomplete mental models, and added interface transparency did not reliably improve those models or satisfaction.
-
J1: Exploring Simple Test-Time Scaling for LLM-as-a-Judge
J1-7B, a judge LLM trained with supervised fine-tuning and reinforcement learning, improves when forced to reflect with 'wait' tokens, and the scaling ability emerges during the RL phase.
-
Improving Human-Autonomous Vehicle Interaction in Complex Systems
Human-AV communication should adapt to context, rider traits, and goals, as shown by two experiments and a machine-learning trust prediction study.
-
What Makes for a Good Saliency Map? Comparing Strategies for Evaluating Saliency Maps in Explainable AI (XAI)
A user study of 166 ICT graduates shows Grad-CAM best improves users' ability to predict classifier accuracy, Guided Backpropagation scores best on mathematical metrics, and the two rankings do not align.
-
tAIfa: Enhancing Team Effectiveness and Cohesion with AI-Generated Automated Feedback
An LLM-powered Slack agent that generates automated team and individual feedback increased conversation duration and speaker turn frequency in a between-subjects study of 18 teams, with no significant effect on task p...
-
Explaining Facial Expression Recognition
FAU-based textual and multimodal explanations improved user understanding and appropriate trust in a facial expression recognition model, but the explanation generator's use of the model's emotion vector as input crea...
-
Explanatory Debiasing: Involving Domain Experts in the Data Generation Process to Mitigate Representation Bias in AI Systems
Involving domain experts in AI data generation can reduce representation bias while maintaining or slightly improving model accuracy, according to a 35-participant healthcare user study.
-
Dynamic Prompt Middleware: Contextual Prompt Refinement Controls for Comprehension Tasks
Users preferred dynamically generated prompt-refinement controls over a fixed preset list when steering AI explanations, reporting more control and lower context-providing barriers, despite difficulty predicting optio...
-
Let people fail! Exploring the influence of explainable virtual and robotic agents in learning-by-doing tasks
People who practiced a simulated task without AI help learned it better than people who got explainable computer or robot assistance.
-
Exploring the Interaction of Explanation Styles, Context, and Trust of AI Privacy Redaction in AI-mediated Interactions
Explanation preferences for AI privacy redaction vary systematically with domain and redaction amount; giving users their preferred styles raises trust over random or no explanations.
-
INSIGHTS: Demonstration-Based Summaries of Time Series Predictors
INSIGHTS creates manageable global summaries of time series model behavior by balancing sample importance and diversity with domain-specific utility functions, validated via experiments and user studies.
-
Evaluation of AI Ethics Tools in Language Models: A Developers' Perspective Case Study
In interviews with 11 Portuguese-language model developers, four AI ethics tools guided general ethical reflection but failed to surface Portuguese-specific harms like cultural misrepresentation and low language performance.
-
MetaExplainer: A Framework to Generate Multi-Type User-Centered Explanations for AI Systems
A neuro-symbolic pipeline decomposes user questions, delegates to model explainers, and synthesizes natural-language explanations, achieving moderate stage-wise scores on a diabetes dataset.
-
Feature-Guided Neighbor Selection for Non-Expert Evaluation of Model Predictions
FGNS ranks neighbor examples by weighted distance to a class prototype, letting non-experts detect AI misclassifications faster and more accurately than with standard k-NN explanations.
-
Video-Mediated Emotion Disclosure: Expressions of Fear, Sadness, and Joy by People with Schizophrenia on YouTube
People with schizophrenia use two video structures (talk-to-camera and in-the-moment), two verbal strategies (direct expression and storytelling), and distinct visual styles to disclose emotions on YouTube, with in-th...
-
Importance of User Control in Data-Centric Steering for Healthcare Experts
Healthcare experts who manually adjusted training data improved a diabetes prediction model more than those using automated corrections, without losing trust or understanding.
-
KERAIA: An Adaptive and Explainable Framework for Dynamic Knowledge Representation and Reasoning
KERAIA is a symbolic AI framework that uses context-sensitive knowledge clouds and explicit reasoning traces to make knowledge representation adaptive and explainable.
-
Towards Counterfactual and Contrastive Explainability and Transparency of DCNN Image Classifiers
The paper learns sparse masks over a DCNN's top-layer filters that preserve the inferred class (contrastive) or flip it to an alter class (counterfactual), evaluated on CUB bird classification.
-
Co-Activation Graph Analysis of Safety-Verified and Explainable Deep Reinforcement Learning Policies
The authors apply co-activation graph analysis to state sets generated by model checking to rank neuron and feature importance in safe deep RL policies.
-
Explainers' Mental Representations of Explainees' Needs in Everyday Explanations
Explainers' assumptions about a listener's knowledge start with technical details (Architecture) and later broaden to include purpose and strategy (Relevance), while interest assumptions follow the opposite path.
-
Mod-Guide: An LLM-based Content Moderation Feedback System to Address Insensitive Speech toward Indigenous Ethnic and Religious Minority Communities
Mod-Guide uses RAG with a community co-created corpus to make LLM moderation responses more contextually accurate for insensitive speech toward Bangladesh's Hindu and Chakma minorities, with mixed-method evaluation sh...
-
Beyond Explainable AI (XAI): An Overdue Paradigm Shift and Post-XAI Research Directions
Current XAI methods for DNNs and LLMs rest on paradoxes and false assumptions that demand a paradigm shift to verification protocols, scientific foundations, context-aware design, and faithful model analysis rather th...
-
Uncertainty Awareness and Trust in Explainable AI- On Trust Calibration using Local and Global Explanations
People who saw uncertainty visualizations trusted a more certain model more, but the study does not demonstrate that this explanation calibrates trust better than numeric accuracy.
-
Minimal Data, Maximum Clarity: A Heuristic for Explaining Optimization
EZR combines active Naive Bayes sampling and decision-tree distillation to reach over 90% of best-known multi-objective optimization performance on 60 datasets while producing clearer explanations than LIME, SHAP or B...
-
Explainable Knowledge Graph Retrieval-Augmented Generation (KG-RAG) with KG-SMILE
KG-SMILE applies perturbation and linear regression to a knowledge graph to attribute which entities and relations drive a GraphRAG system's answers.
-
A Survey of Explainable Reinforcement Learning: Targets, Methods and Needs
A survey of 250+ explainable-reinforcement-learning papers proposes a What/How taxonomy and reports that sequence-level explanations are rare (11 works) compared with policy-level (175) and action-level (89) ones.
-
Human-Centered Explainability in Interactive Information Systems: A Survey
A systematic review of 100 empirical user studies synthesizes explainability research into five conceptual dimensions, a design classification, and six measurement categories.
-
A Systematic Review of User-Centred Evaluation of Explainable AI in Healthcare
A systematic review of 82 healthcare XAI user studies produces an updated property framework and context-sensitive guidelines for evaluation design.
-
Gap the (Theory of) Mind: Sharing Beliefs About Teammates' Goals Boosts Collaboration Perception, Not Performance
In a human-AI teamwork game, showing the AI's inferred goals improved players' perceived understanding of the teammate but not objective performance or overall satisfaction.
-
DiCE-Extended: A Robust Approach to Counterfactual Explanations in Machine Learning
DiCE-Extended modifies the DiCE counterfactual explainer by adding a Dice-Sørensen-based robustness loss, claiming improved stability and proximity across four tabular datasets.
-
Explainability for Embedding AI: Aspirations and Actuality
Survey evidence shows developers want XAI support, yet most participants using LIME or SHAP failed to detect deliberately injected data faults in a housing price model.
-
Objective Metrics for Human-Subjects Evaluation in Explainable Reinforcement Learning
For evaluating explanations in reinforcement learning, this paper recommends measuring observable human behavior, such as predicting the agent's next action or goal, instead of relying only on subjective ratings.
-
Meta-evaluating stability measures: MAX-Senstivity & AVG-Sensitivity
MAX-Sensitivity and AVG-Sensitivity report near-perfect stability on randomly generated explanations and predictions, failing a simple random-output sanity test.
-
Explainable Optimization: A Call for Interdisciplinary Action
The paper calls for establishing explainable optimization (XOpt) as an interdisciplinary area to bridge the gap between optimization outputs and stakeholder needs for justification.
-
Towards Transparent Ethical AI: A Roadmap for Trustworthy Robotic Systems
The paper argues transparency is fundamental to trustworthy robotics and proposes a framework connecting technical transparency tools to ethical outcomes such as accountability and informed consent.
-
Adopting Trustworthy AI for Sleep Disorder Prediction: Deep Time Series Analysis with Temporal Attention Mechanism and Counterfactual Explanations
LSTM, TCN, and TFT predict sleep disorders from a 400-patient dataset with reported accuracies between 85 and 93 percent, and SHAP, temporal attention, and counterfactuals are offered as interpretability aids.
-
Explainable Artificial Intelligence for Medical Applications: A Review
A review that categorizes XAI techniques and surveys medical applications across vision, audio, and multimodal data, with an analysis of current challenges.
-
ClarifAI: Enhancing AI Interpretability and Transparency through Case-Based Reasoning and Ontology-Driven Approach for Improved Decision-Making
ClarifAI is a conceptual architecture combining case-based reasoning and ontology-driven knowledge to generate explanations for AI decisions, presented without implementation or testing.
-
A Brief Summary of Explanatory Virtues
A concise literature survey that organizes explanatory virtues into Keas's four categories plus a new coverage category, and relates them to explainable AI.
Discussion (0). Continue with ORCID to comment.