Recognition: no theorem link
How Can Explainable Artificial Intelligence Improve Trust and Transparency in Medical Diagnosis Systems?
Pith reviewed 2026-05-15 11:32 UTC · model grok-4.3
The pith
Explanations in AI medical diagnosis systems boost trust and perceived usefulness based on a student survey.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The authors claim that providing explanations significantly increases trust, clarity, and perceived safety of AI recommendations in medical diagnosis. Knowledge of XAI showed a positive correlation with trust (r = 0.48, p = 0.01) and perceived usefulness (r = 0.60, p = 0.001). They conclude that explainability is a key factor for successful integration of AI in healthcare decision support systems, with participants preferring AI as a supportive tool rather than a replacement for human judgment.
What carries the argument
The structured survey assessing the influence of XAI understanding on confidence in AI decisions, perceived usefulness, and adoption intentions.
If this is right
- Explanations improve transparency and trust in AI-assisted medical tools.
- XAI knowledge positively correlates with trust and usefulness ratings.
- AI systems should be designed to function as support tools alongside human clinicians.
- Explainability facilitates better integration of AI into healthcare decision-making.
Where Pith is reading between the lines
- Medical education programs could include XAI training to improve future adoption of diagnostic AI.
- Similar surveys with practicing physicians might reveal different patterns in real-world settings.
- Testing these effects in live clinical environments could confirm if explanations actually reduce diagnostic errors.
Load-bearing premise
That the self-reported responses of 30 medical students to a survey accurately capture how explanations would affect trust and behavior in real clinical practice.
What would settle it
A randomized controlled trial where actual doctors diagnose cases using AI with and without explanations and measure changes in trust, reliance, and diagnostic accuracy.
read the original abstract
The growing adoption of artificial intelligence in healthcare has raised concerns about the transparency and trustworthiness of AI-driven medical diagnosis systems. Many existing models operate as black boxes, limiting clinicians' ability to understand how decisions are made. Explainable Artificial Intelligence (XAI) has been proposed as a solution to improve transparency, interpretability, and trust in AI-assisted medical tools. This study investigates the relationship between explainability and trust in AI-based diagnostic systems. A structured survey of 30 medical students was conducted to examine the influence of XAI understanding, confidence in AI decisions, perceived usefulness, and adoption intentions. The results indicate that explanations significantly increase trust, clarity, and perceived safety of AI recommendations. Knowledge of XAI showed a positive correlation with trust (r = 0.48, p = 0.01) and perceived usefulness (r = 0.60, p = 0.001). The findings suggest that explainability is a key factor for successful integration of AI in healthcare decision support systems. While AI explanations improve transparency and trust, participants still prefer AI to function as a support tool rather than replacing human clinical judgment.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript presents an empirical study based on a survey of 30 medical students to investigate how explainable artificial intelligence (XAI) can enhance trust and transparency in AI-driven medical diagnosis systems. Key findings include positive correlations between XAI knowledge and trust (r = 0.48, p = 0.01) as well as perceived usefulness (r = 0.60, p = 0.001), leading to the conclusion that explanations improve perceived safety and that AI should function as a supportive tool alongside human judgment.
Significance. Should the reported correlations hold under more rigorous validation, the work would provide useful preliminary evidence supporting the integration of XAI techniques in healthcare AI systems to foster clinician trust. This aligns with broader efforts in human-computer interaction to address black-box concerns in high-stakes domains like medicine.
major comments (2)
- [Abstract and Results] The central claim that XAI explanations significantly increase trust and perceived usefulness rests on a cross-sectional survey of 30 medical students using only self-reported measures. No details are supplied on survey validation, sampling method, controls, or behavioral outcomes such as adherence to AI recommendations in actual diagnostic tasks (Abstract and Results sections).
- The sample is restricted to medical students rather than practicing clinicians, and the design appears vignette- or hypothetical-based rather than involving real interaction with an AI diagnostic model with/without explanations. This undermines generalizability to clinical trust and adoption (Methods and Discussion sections).
minor comments (2)
- [Abstract] The abstract asserts that 'explanations significantly increase trust' without specifying the exact comparison (e.g., with vs. without explanations) or reporting confidence intervals around the correlation coefficients.
- [Methods] No power analysis or justification for the sample size of 30 is provided, and the manuscript does not discuss potential response biases or prior AI exposure as covariates.
Simulated Author's Rebuttal
We thank the referee for the constructive comments, which have helped us strengthen the methodological transparency and limitations discussion in the manuscript. We have revised the text to address the concerns about missing details and generalizability while preserving the scope of this preliminary survey study.
read point-by-point responses
-
Referee: [Abstract and Results] The central claim that XAI explanations significantly increase trust and perceived usefulness rests on a cross-sectional survey of 30 medical students using only self-reported measures. No details are supplied on survey validation, sampling method, controls, or behavioral outcomes such as adherence to AI recommendations in actual diagnostic tasks (Abstract and Results sections).
Authors: We agree that additional methodological details were needed. The revised Methods section now describes survey development (including pilot testing for clarity and face validity with five medical students), the convenience sampling procedure for recruiting the 30 participants, and controls for prior XAI knowledge via a screening question. We have also added explicit statements that the study relied on self-reported measures and did not assess behavioral outcomes such as diagnostic adherence; this is framed as a limitation with directions for future experimental work. revision: yes
-
Referee: The sample is restricted to medical students rather than practicing clinicians, and the design appears vignette- or hypothetical-based rather than involving real interaction with an AI diagnostic model with/without explanations. This undermines generalizability to clinical trust and adoption (Methods and Discussion sections).
Authors: We acknowledge that restricting the sample to medical students limits direct applicability to practicing clinicians and have expanded the Discussion to emphasize this point, including a call for replication with experienced physicians. The study was designed as an exploratory survey to capture perceptions and correlations rather than a controlled experiment with live AI interaction; we have clarified this intent in Methods and noted the hypothetical nature of the vignettes as a boundary condition on the findings. revision: partial
Circularity Check
No significant circularity in empirical survey study
full rationale
The paper is a purely empirical investigation using a structured survey of 30 medical students. It reports correlations such as r = 0.48 (p = 0.01) between XAI knowledge and trust directly from the collected data. No equations, derivations, or model-based predictions are present. There are no self-citations that bear the load of any central claim, no uniqueness theorems invoked, and no ansatzes or renamings. The findings are straightforward data outcomes, making the study self-contained without circularity.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption Self-reported perceptions in a structured survey of medical students validly measure trust, clarity, and adoption intentions toward AI systems.
Reference graph
Works this paper leans on
-
[1]
P. A. Moreno-Sánchez, “Data-driven early diagnosis of chronic kidney disease: Development and evaluation of an explainable AI model,” IEEE Access, vol. 11, pp. 38359–38369, 2023, doi: 10.1109/ACCESS.2023.10091536. [4] T. Räz, A. Pahud De Mortanges, and M. Reyes, “Explainable AI in medicine: Challenges of integrating XAI into the future clinical routine,” ...
-
[2]
A unified approach to interpreting model predictions in healthcare,
J. F. Hair Jr., Essentials of Multivariate Data Analysis, Cengage, 2020. (О Pearson correlation.) [18] J. K. Lundberg and S. Lee, “A unified approach to interpreting model predictions in healthcare,” IEEE Transactions on Medical Imaging, vol. 39, no. 11, pp. 4505–4515, 2020. [19] F. Holzinger et al., “Human-in-the-loop explainable AI for medical applicati...
work page 2020
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.