A narrative review arguing that medical LLM evaluations should examine reasoning behaviour, not only accuracy, and proposing two conceptual transparency frameworks.
Doctor XAvIer: Explainable Diagnosis on Physician-Patient Dialogues and XAI Evaluation
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
We introduce Doctor XAvIer, a BERT-based diagnostic system that extracts relevant clinical data from transcribed patient-doctor dialogues and explains predictions using feature attribution methods. We present a novel performance plot and evaluation metric for feature attribution methods: Feature Attribution Dropping (FAD) curve and its Normalized Area Under the Curve (N-AUC). FAD curve analysis shows that integrated gradients outperforms Shapley values in explaining diagnosis classification. Doctor XAvIer outperforms the baseline with 0.97 F1-score in named entity recognition and symptom pertinence classification and 0.91 F1-score in diagnosis classification.
citation-role summary
method 1
citation-polarity summary
fields
cs.CL 1years
2024 1verdicts
CONDITIONAL 1roles
method 1polarities
support 1representative citing papers
citing papers explorer
-
Critique of Impure Reason: Unveiling the reasoning behaviour of medical Large Language Models
A narrative review arguing that medical LLM evaluations should examine reasoning behaviour, not only accuracy, and proposing two conceptual transparency frameworks.