Pith. sign in

REVIEW 5 major objections 4 minor 13 references

LLM Sensitivity Evaluation Framework for Clinical Diagnosis

T0 review · 5 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Current LLMs fail to stay sensitive to diagnosis-changing medical details; GPT-4 answers only 5.28% of altered cases correctly.

desk verdict A useful new sensitivity benchmark for clinical LLMs, but the headline DAS number rests on unvalidated human labels and Section 6.4 has a swapped-numbers error. read the letter →

arxiv 2504.13475 v1 pith:HYAP7RGQ submitted 2025-04-18 cs.CL

Chenwei Yan , Xiangling Fu , Yuxuan Xiong , Tianyi Wang , Siu Cheung Hui , Ji Wu , Xien Liu This is my paper · ORCID
classification cs.CL
keywords LLMsensitivityevaluationclinicaldiagnosiskeymedicalinformationperturbationstrategiesQAMedmultiple-choicequestionansweringGPT-4reliability
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether large language models behave like careful physicians when a small but diagnosis-relevant detail changes. To answer it, the authors build DiagnosisQA from medical exam questions and create eight derived datasets that change or remove four kinds of key medical information: age, gender, symptoms, and checkup results. The central finding is that all five tested models are unreliable on this test: GPT-4, the best, answers only 345 of 6,528 questions correctly (5.28%) when the perturbation changes the correct diagnosis, and several models also lose accuracy on changes that should leave the diagnosis unchanged. The paper argues that accuracy on standard medical benchmarks does not measure clinical reliability, and that LLM development should aim for explicit sensitivity to key information.

What carries the argument

The load-bearing mechanism is a perturbation matrix: four types of key medical information ($K = \{gender, age, symptom, checkup\}$) crossed with two perturbation strategies (change and removal), producing eight derived datasets. Each perturbed question is then classified as Same Answer Subset (SAS) or Different Answer Subset (DAS) through physician re-annotation, and two extra options—'None of the above' and 'The question contains inconsistency'—are added when the perturbation removes the correct option or creates a clinically impossible case. Sensitivity is read as the accuracy difference on SAS before and after perturbation (should be near zero) and accuracy on DAS (should be high). The framework also tracks response rate and instruction-following rate to separate format compliance from diagnostic sensitivity.

What would settle it

Have an independent panel of physicians re-annotate a random sample of, say, 300 questions from the Different Answer Subset without seeing the paper's labels, and measure agreement on the correct option; if agreement falls below roughly 90%, the reported 5.28% GPT-4 accuracy is contaminated by label noise and is not a clean measure of LLM sensitivity.

Watch

Extended reading notes

Core claim

The paper's central claim is that current LLMs do not have the sensitivity profile a clinician needs: they should notice when key information changes the diagnosis, and stay stable when it does not, but they do neither reliably. The evidence is a set of eight derived datasets in which each question's gender, age, symptom, or checkup result is either changed or removed, with correct answers re-annotated by physicians. Questions are split into a Same Answer Subset, where the diagnosis is unchanged, and a Different Answer Subset, where the correct answer changes. GPT-4 answers only 345 of 6,528 DAS questions correctly (5.28%), far above the other models but still far too low for clinical use, while models such as Gemini are easily thrown off by symptom changes that should not alter the diagnosis. From this the paper concludes that high performance on medical licensing benchmarks is not evidence of clinical reliability.

Load-bearing premise

The measurement stands or falls on the physician re-annotations of the eight derived datasets being correct and complete, and on the perturbation-and-extraction rules (including the added inconsistency option) producing exactly one valid correct answer for every question.

Editorial extensions

If this is right

  • Medical LLM evaluation should include sensitivity stress tests, not only accuracy on unperturbed exam questions.
  • A high score on MedQA-style benchmarks does not imply a model will notice when a patient's gender, symptom, or test result changes the diagnosis.
  • Models need two opposite skills at once: stability on diagnosis-irrelevant changes and responsiveness to diagnosis-relevant ones; current models fail at one or both.
  • Instruction-following and sensitivity are separate failure modes: LLaMA2-7b has a 0% followed-instruction rate, while GPT-4 follows instructions almost perfectly yet still misses 94.72% of DAS questions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the reported five models, the same datasets could be used to test whether medical fine-tuning or larger scale narrows the DAS gap, since the paper does not show that any training recipe repairs it.
  • The 5.28% figure likely mixes two distinct failures—not noticing that a key fact changed, and noticing but failing to update the diagnosis—and a follow-up that asks models to first state which fact changed would separate them.
  • The perturbation design could be recycled as a training signal: exposing models to DAS examples with the corrected labels might improve sensitivity without requiring new data collection.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper proposes LLMSenEval, a framework for evaluating the sensitivity of large language models to key medical information in clinical diagnosis. The authors construct DiagnosisQA, a set of 4,603 case-based multiple-choice questions derived from MedQA, and generate eight derived datasets by applying two perturbation strategies (change and removal) to four types of key information (gender, age, symptoms, checkup results). They evaluate GPT-3.5, GPT-4, Gemini, Claude3, and LLaMA2-7b on these datasets, partitioning each derived dataset into Same Answer Subset (SAS) and Different Answer Subset (DAS) based on physician annotation. The central empirical claim is that current LLMs, including the best-performing GPT-4, show poor sensitivity to key medical information: GPT-4 answers only 345 of 6,528 DAS questions correctly (5.28%). The paper concludes that LLMs are not yet reliable for clinical diagnosis because they fail to update diagnoses when key information is perturbed.

Significance. The work addresses a genuine and underexplored gap: existing sensitivity evaluations of LLMs focus on irrelevant context and option-order artifacts, whereas clinical reliability depends on sensitivity to clinically meaningful changes. The proposed framework is systematic, the dataset and code are publicly released, multiple LLMs are compared, and the authors include a thoughtful limitations section. If the ground-truth annotations for the perturbed datasets are valid, the reported low DAS accuracies constitute a meaningful and cautionary empirical finding for the medical AI community. The framework itself is reusable and could be extended to other key information types and perturbation strategies. The paper is not circular: the evaluation criteria are defined independently of model behavior, and the numerical claims are falsifiable against the released dataset.

major comments (5)
  1. [Section 6.4, Table 7] There is an internal numeric inconsistency in the GPT-4 checkup DAS results. The text states that GPT-4 correctly answers 71 and 22 questions from the 1,720 checkup-change and 411 checkup-removal questions, 'with an accuracy of 3.08% and 3.65%, respectively,' but Table 7 reports GPT-4 DAS accuracies of 4.13% for change and 5.25% for removal. The correct percentages from the stated counts are 71/1720 = 4.13% and 22/411 = 5.35%, so the table value 5.25% is also slightly off. This inconsistency affects the reported headline numbers and must be corrected; the text and the table should be reconciled and a single consistent set of values used throughout.
  2. [Section 4.1 and Section 6 (annotation reliability)] The DAS ground-truth labels are load-bearing because every DAS accuracy, including the headline 5.28% (345/6,528), depends on the correctness of the physician relabeling of perturbed questions. The paper reports only that 'four professional physicians and experts' reviewed and corrected the derived datasets; it gives no inter-annotator agreement, no number of independent annotations per item, no conflict-resolution procedure, and no adjudication details. Without this information, a nontrivial fraction of DAS labels could be annotator-dependent, and the low accuracies could partly reflect label noise or ambiguous questions rather than LLM insensitivity. Please report annotation reliability statistics (e.g., Cohen's kappa or Fleiss' kappa) and describe the labeling protocol.
  3. [Section 6, DAS composition and meta-options] The DAS mixes two different types of questions: those where the perturbation changes the correct diagnosis to another medical option, and those where the correct label is a meta-option ('None of the above' or 'The question contains inconsistency'). For example, the gender-change DAS has 378 items and GPT-4 answers 121 correctly, but many of these are cases where the model must recognize a clinically impossible scenario (e.g., a male patient with menstruation) rather than reason to an alternative diagnosis. The aggregated 5.28% conflates these abilities, and the low accuracy may partly reflect models' difficulty with meta-options rather than insensitivity to diagnostic information. Please report DAS accuracies separately for diagnosis-change items and meta-option items, or justify why pooling is appropriate.
  4. [Section 3.2 and Table 2 (perturbation extraction and dataset filtering)] The regex/keyword extraction described in Section 3.2 determines which questions are included in each derived dataset and which key-information value is perturbed, but no validation of this extraction is reported. The derived datasets exclude questions that could not be perturbed due to missing key information, yet the manuscript does not quantify how many questions were excluded per derived dataset or whether the exclusions introduce selection bias in the SAS/DAS partition. Please provide precision/recall or a manual review sample for the extraction, and report the exclusion counts and their overlap across the eight derived datasets.
  5. [Section 6.1 and 6.2 (small DAS subsets)] Several DAS cells contain only a handful of items (gender removal n=4, age change n=4, age removal n=22), yet the text draws comparative conclusions from these, e.g., 'Claude3 achieves the highest sensitivity to age removal' based on one correct answer out of 22. These percentages have very wide confidence intervals and are not statistically reliable. The central claim about LLM insensitivity should be based primarily on the aggregate DAS or the larger subsets, and the small-sample cells should be explicitly labeled as unreliable or removed from comparative claims.
minor comments (4)
  1. [Section 6.4, text vs Table 7] The paragraph on checkup SAS results says GPT-4 exhibits 'an increase of 0.37% in ∆ accuracy for checkup changes,' but Table 7 shows GPT-4's SAS ∆ accuracy as 0 for change and +0.37 for removal. The text and table contradict each other; please correct the description.
  2. [Table 1 and Section 3.2] The derived dataset abbreviations (DGC, DGR, DAC, DAR, DSC, DSR, DCC, DCR) are introduced in Section 3.2 but Table 1 lists them without a separate legend; adding a short expansion or one-sentence definition in the caption would improve readability.
  3. [Section 4.3] Equation (3) defines FIR as #followedInstructionR / #validR, but the text states LLaMA2-7b has FIR=0 while its RR is high; it would be clearer to state explicitly whether plain-text responses (which are valid but not instruction-following) are the cause of the zero FIR.
  4. [Section 7 and Figure 4] The discussion of 'low sensitivity' vs 'high sensitivity' is somewhat confusing because SAS accuracy stability and DAS accuracy are opposite notions of sensitivity. A short definition of the intended semantics before the discussion would help readers interpret the figure and the qualitative claims.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the evaluation is an externally grounded benchmark with no fitted parameters or load-bearing self-citations.

full rationale

The paper's derivation chain is an empirical evaluation pipeline: DiagnosisQA is filtered from MedQA, eight derived datasets are produced by fixed perturbation strategies (gender/age symptom/checkup change or removal), and the Same Answer Subset / Different Answer Subset split is assigned by physician annotation rather than by the evaluated models. The sensitivity metric is computed directly as accuracy on the physician-labeled DAS questions and the change in accuracy on SAS questions, with no fitted parameter, learned rescaling, or ansatz whose value is tuned to produce the headline result. GPT-4's 5.28% DAS accuracy is a measured outcome, not an output-equivalent input. The only self-citation is Zhou et al. (2024), cited in Section 2.1 as one of several works on reliability and interpretability; it is not load-bearing for the framework, dataset construction, or conclusions. No equation is defined in terms of a result it later predicts, and no uniqueness claim is imported from prior work. Possible annotation noise or perturbation validity concerns are correctness/validity risks, not circularity.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The evaluation framework relies on design choices such as the four key information types, the change/removal perturbation strategies, and physician-labeled ground truth. No fitted model parameters are introduced, but the arbitrary age perturbation magnitude and the untested assumptions about ground truth validity are the main burdens.

free parameters (1)
  • age perturbation magnitude = 20%
    Hand-chosen threshold for age change; different values might yield different sensitivity measurements, and the 20% figure is not justified by clinical reasoning.
assumptions (4)
  • domain assumption Physician re-annotation of perturbed questions is correct
    Section 4.1 states derived datasets were annotated with help of four professional physicians, but no inter-annotator agreement is reported; if labels are wrong, sensitivity metrics are invalid.
  • domain assumption MedQA case-based questions approximate real clinical diagnosis
    The paper acknowledges in Section 9 that QA questions differ from real EMRs, so the domain transfer is an assumption.
  • domain assumption The perturbation strategies capture clinically meaningful variation
    Table 1 defines the perturbations; some, like gender swap in sex-specific diseases, create impossible cases rather than realistic clinical variation.
  • standard math Standard statistical calculations are valid
    The SE values in Tables 4-7 are used to interpret delta accuracy; their derivation is not stated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LLM Sensitivity Evaluation Framework for Clinical Diagnosis." pith.science (2026). https://pith.science/paper/HYAP7RGQ

@misc{pith2026250413475,
  author       = {Pith},
  title        = {Pith review of: LLM Sensitivity Evaluation Framework for Clinical Diagnosis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HYAP7RGQ}},
  note         = {Machine review of arXiv:2504.13475}
}
read the original abstract

Large language models (LLMs) have demonstrated impressive performance across various domains. However, for clinical diagnosis, higher expectations are required for LLM's reliability and sensitivity: thinking like physicians and remaining sensitive to key medical information that affects diagnostic reasoning, as subtle variations can lead to different diagnosis results. Yet, existing works focus mainly on investigating the sensitivity of LLMs to irrelevant context and overlook the importance of key information. In this paper, we investigate the sensitivity of LLMs, i.e. GPT-3.5, GPT-4, Gemini, Claude3 and LLaMA2-7b, to key medical information by introducing different perturbation strategies. The evaluation results highlight the limitations of current LLMs in remaining sensitive to key medical information for diagnostic decision-making. The evolution of LLMs must focus on improving their reliability, enhancing their ability to be sensitive to key information, and effectively utilizing this information. These improvements will enhance human trust in LLMs and facilitate their practical application in real-world scenarios. Our code and dataset are available at https://github.com/chenwei23333/DiagnosisQA.

Figures

Figures reproduced from arXiv: 2504.13475 by the authors.

Figure 1
Figure 1. Human doctors are sensitive to key medical [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The proposed framework LLMSenEval for LLM sensitivity evaluation. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Valid and invalid responses from LLMs. Dataset # Total # 5-Option # 4-Option DiagnosisQA 4,603 3,696 907 DGC, DGR 3,965 3,236 729 DAC, DAR 4,008 3,342 666 DSC, DSR 3,463 2,731 732 DCC, DCR 3,439 2,903 536 [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: The overall sensitivity performance of five LLMs. The bar chart shows the average difference in accuracy [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: The impact of prompts on the performance of [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

13 extracted references · 1 canonical work pages

  1. [4]

    Preprint, arXiv:2302.14520

    Large language models are state-of-the-art evaluators of translation quality. Preprint, arXiv:2302.14520. Cheng Li, Jindong Wang, Yixuan Zhang, Kaijie Zhu, Wenxin Hou, Jianxun Lian, Fang Luo, Qiang Yang, and Xing Xie

    arXiv

  2. [5]

    Preprint, arXiv:2307.11760

    Large language models un- derstand and can be enhanced by emotional stimuli. Preprint, arXiv:2307.11760. Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. 2023a. Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation. Preprint, arXiv:2305.01210. Junling Liu, Peilin Zhou, Yining Hua,...

    arXiv

  3. [6]

    Preprint, arXiv:2303.08774

    Gpt-4 technical report. Preprint, arXiv:2303.08774. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Car- roll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe

    arXiv

  4. [8]

    arXiv preprint arXiv:2308.11483

    Large language models sensitivity to the order of options in multiple-choice questions. arXiv preprint arXiv:2308.11483. Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed H Chi, Nathanael Schärli, and Denny Zhou

    arXiv

  5. [9]

    Preprint, arXiv:2307.09288

    Llama 2: Open foundation and fine- tuned chat models. Preprint, arXiv:2307.09288. Rongwu Xu, Brian S Lin, Shujian Yang, Tianqi Zhang, Weiyan Shi, Tianwei Zhang, Zhixuan Fang, Wei Xu, and Han Qiu

    arXiv

  6. [10]

    arXiv preprint arXiv:2312.09085

    The earth is flat be- cause...: Investigating llms’ belief towards misinfor- mation via persuasive conversation. arXiv preprint arXiv:2312.09085. Xinlu Zhang, Shiyang Li, Xianjun Yang, Chenxin Tian, Yao Qin, and Linda Ruth Petzold

    arXiv

  7. [11]

    Enhancing Small Medical Learners with Privacy-preserving Contextual Prompting

    Enhancing small medical learners with privacy-preserving con- textual prompting. arXiv preprint arXiv:2305.12723. Chujie Zheng, Hao Zhou, Fandong Meng, Jie Zhou, and Minlie Huang

    work page Pith review arXiv

  8. [12]

    arXiv preprint arXiv:2309.03882

    On large language models’ se- lection bias in multi-choice questions. arXiv preprint arXiv:2309.03882. Yuxuan Zhou, Xien Liu, Chen Ning, and Ji Wu

    arXiv

Show all 13 references
  1. [13]

    Preprint, arXiv:2406.02919

    Multifaceteval: Multifaceted evaluation to probe llms in mastering medical knowledge. Preprint, arXiv:2406.02919. A The Impact of Prompts To investigate the effects of prompts, we conduct a comparative experiment on the DiagnosisQA with empty prompt in the model input, but kee...

    arXiv

  2. [2020]

    Preprint, arXiv:2005.14165

    Language models are few-shot learn- ers. Preprint, arXiv:2005.14165. Sondos Mahmoud Bsharat, Aidar Myrzakhan, and Zhiqiang Shen

    2005 arXiv

  3. [2022]

    Preprint, arXiv:2203.02155

    Training language models to follow instructions with human feedback. Preprint, arXiv:2203.02155. Pouya Pezeshkpour and Estevam Hruschka

    arXiv

  4. [2023]

    Preprint, arXiv:2312.12806

    Med- bench: A large-scale chinese benchmark for eval- uating medical large language models. Preprint, arXiv:2312.12806. Wendy W. Chapman, Will Bridewell, Paul Hanbury, Gregory F. Cooper, and Bruce G. Buchanan

    arXiv

  5. [2024]

    Preprint, arXiv:2312.16171

    Principled instructions are all you need for questioning llama-1/2, gpt-3.5/4. Preprint, arXiv:2312.16171. Yan Cai, Linlin Wang, Ye Wang, Gerard de Melo, Ya Zhang, Yanfeng Wang, and Liang He

    arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.