A new German open-response medical benchmark shows LLM judges can match physician agreement (κ≈0.69 vs 0.71) yet lack clinical caution and show lineage-dependent scoring bias.
Evaluating the performance of GPT-3.5, GPT-4, and GPT-4o in the Chinese National Medical Licensing Examination
1 Pith paper cite this work, alongside 27 external citations. Polarity classification is still indexing.
1
Pith paper citing it
27
external citations · OpenAlex
fields
cs.CL 1years
2026 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Clinician-Level Agreement Without Clinical Caution: LLM Evaluator Limits in Medical AI Benchmarking
A new German open-response medical benchmark shows LLM judges can match physician agreement (κ≈0.69 vs 0.71) yet lack clinical caution and show lineage-dependent scoring bias.