Domain specialization does not consistently improve clinical LLM robustness to meaning-preserving prompt variations, as shown by new sensitivity metrics on DiagnosisQA and MedQA.
arXiv preprint arXiv:2306.11270 , year=
3 Pith papers cite this work. Polarity classification is still indexing.
3
Pith papers citing it
fields
cs.CL 3years
2026 3representative citing papers
Introduces AIR, an asymmetric regularization that anchors open-ended safety prompts to verifiable ones via stop-gradient, improving invariance and accuracy when combined with group preference optimization.
Counterfactual prompting effects on LLMs are often indistinguishable from those caused by meaning-preserving paraphrases, causing most previously reported demographic sensitivities to disappear under proper statistical comparison.
citing papers explorer
-
Same Patient, Different Words, Different Diagnosis? Evaluating Semantic Stability in Clinical LLMs
Domain specialization does not consistently improve clinical LLM robustness to meaning-preserving prompt variations, as shown by new sensitivity metrics on DiagnosisQA and MedQA.
-
Towards Context-Invariant Safety Alignment for Large Language Models
Introduces AIR, an asymmetric regularization that anchors open-ended safety prompts to verifiable ones via stop-gradient, improving invariance and accuracy when combined with group preference optimization.
-
Compared to What? Baselines and Metrics for Counterfactual Prompting
Counterfactual prompting effects on LLMs are often indistinguishable from those caused by meaning-preserving paraphrases, causing most previously reported demographic sensitivities to disappear under proper statistical comparison.