REVIEW 3 major objections 5 minor 34 references
Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning
T0 review · 3 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash
Pith's one-line read The paper claims that LLM medication-safety accuracy on fixed patient cases (63.6%) drops to 45.1% on paired counterfactual questions where the same rule no longer applies, and that models often acknowledge the changed patient information…
desk verdict Worth reading and worth reviewing: the benchmark is solid and the activation–deactivation asymmetry is a real failure mode, but the headline GF–CF gap is inflated by comparing unmatched question sets. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the paired guideline-following/counterfactual question design produced by the MedRule2Pair pipeline. Each verified medication-safety rule is converted into two questions that share the same vignette, answer options, and question stem, but differ in one patient attribute that changes whether the rule applies. This pair design turns counterfactual sensitivity into a measurable quantity: the GF minus CF accuracy gap, the activation-versus-deactivation accuracy split, and pair accuracy over the 89 linked pairs, where a pair counts correct only if both members are right. The controlled reversal of rule applicability is what lets the benchmark attribute failures to conditional rule application rather than to a lack of drug knowledge.
What would settle it
Run the same 28 models on a matched control set of pairs in which the patient information changes but the correct answer stays the same; if the accuracy drop from the guideline-following version matches the observed 18.5-point GF–CF gap, the decline would stem from sensitivity to any vignette change, not specifically from failing to withdraw the rule when it no longer applies.
Extended reading notes
Core claim
The central discovery is that medication-safety accuracy in fixed patient cases overstates a model's ability to reason about when a safety rule applies. On paired counterfactual questions where the same rule no longer applies because a specific patient attribute changed, every one of the 28 evaluated models scored worse, with the mean GF–CF gap at 18.5 points. The failure is directional: models activate a warning triggered by patient information much better (57.7% mean accuracy) than they withdraw it (37.1%), and trigger-only errors are more than twice as frequent as control-only errors in matched pairs. Rationale analysis shows that the problem is not noticing the change: models mention the changed patient information in the large majority of incorrect deactivation responses but still retain the old medication selection. The paper interprets this as evidence that models rely on familiar drug-risk associations rather than binding each judgment to the specific condition that makes the rule apply.
Load-bearing premise
The argument assumes that the paired counterfactual format is not itself harder than the guideline-following format, so the observed accuracy drop measures rule-applicability updating rather than question-format difficulty.
Editorial extensions
If this is right
- Model rankings on static medication-safety questions will not match rankings on counterfactual questions, so leaderboard comparisons that use fixed cases can misrepresent patient-specific reliability.
- Even the best model solves only 48.3% of linked pairs, so high overall accuracy can coexist with poor consistency across matched cases where a rule's applicability flips.
- Deployment of medical LLMs for prescribing support should require evidence of conditional updating, not just accuracy on fixed cases, because a model that keeps a warning after its trigger disappears could harm the patient it is supposed to help.
- Training or prompting that makes models recheck the triggering condition before producing the final answer is a plausible remedy, since many incorrect deactivation responses already contain the correct rule-consistent judgment.
- Medical-specific adaptation can raise absolute counterfactual accuracy but does not eliminate the activation–deactivation asymmetry, meaning specialization alone will not fix rule-applicability control.
Reading between the lines
- A natural control the paper does not run is a matched set of pairs in which patient information changes but the correct answer stays fixed; without it, part of the GF–CF gap could be attributed to sensitivity to any vignette change rather than specifically to rule withdrawal.
- The paired design could transfer to other conditional clinical decisions, such as drug–drug interaction checks, imaging referral rules, or lab-value-based dose adjustments, where the same recognize-but-do-not-update failure may appear.
- Because deactivation errors often contain a correct intermediate judgment, model rationales could be used as a self-check at inference time: when the final answer contradicts a stated rule-consistent option, the model could be prompted to reconcile the two.
- The benchmark measures zero-shot behavior; a plausible next experiment is to test whether few-shot examples of rule deactivation or prompts that explicitly restate the triggering condition reduce the asymmetry.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces MedPIC-Bench, a source-grounded benchmark for medication-safety reasoning, and MedRule2Pair, a two-stage pipeline that converts clinical recommendations (AGS Beers Criteria, PLLR, TGA, KIDs List) into structured rules and paired guideline-following (GF) and counterfactual (CF) questions. The benchmark contains 467 questions annotated along six dimensions. The authors evaluate 28 medical-specific, general, and proprietary LLMs with zero-shot exact-match scoring. They report that mean accuracy drops from 63.6% on GF questions to 45.1% on CF questions, with every model declining; that activation accuracy (57.7%) exceeds deactivation accuracy (37.1%); that mean pair accuracy over 89 linked pairs is only 20.0%; and that in 69.8% of incorrect deactivation responses the model retains the same medication selection despite mentioning the changed patient information in 85.6% of those responses. The paper concludes that static medication-safety accuracy overstates a model's ability to apply conditional safety rules to the patient at hand.
Significance. If the central interpretation holds, the benchmark fills a real gap: existing medication-safety evaluations mostly test fixed scenarios and can be solved by risk recall rather than by conditional rule application. The paper's strengths include source-verifiable ground truth anchored to external clinical references, expert validation with 85.3% initial agreement, cluster-bootstrap confidence intervals for principal aggregates, a deterministic generation pipeline that supports reproducibility, and an honest representative failure case. The benchmark design also yields a falsifiable prediction—models preserve warnings more readily than they retract them—that is worth testing in future work. However, the load-bearing claim that the GF–CF gap measures sensitivity to patient-information changes is not yet fully supported because the headline comparison uses compositionally unmatched question sets and lacks a matched control for the CF format itself.
major comments (3)
- [§4.2 and Fig. 3] The headline 18.5-point GF–CF gap is computed over all 284 GF and 183 CF questions, not over the 89 linked pairs. The two pools are compositionally different: 60.8% of GF items are Static Risk Recognition, whereas CF items are dominated by Risk Activation, Risk Deactivation, Interaction Identification, and Risk Redistribution, which require different and arguably harder reasoning operations. To support the claim that the gap measures sensitivity to patient-information changes, the authors should report mean GF and CF accuracy restricted to the 89 paired items, and ideally per-operation paired accuracies. Without this number, a substantial part of the aggregate gap could reflect the different operation mix rather than counterfactual updating.
- [§3.2] The construction description states that non-target clinical details are preserved 'whenever possible,' but no control condition is provided in which patient information changes while the target rule's applicability and the gold answer remain fixed. Such a condition is needed to separate failures to update rule applicability from generic effects of the CF format, including reversed question framing and the presence of 'None of the above' options. The 67 matched activation–deactivation pairs in Figure 4(b) are informative, but they are not a format control because both members are CF questions. I recommend adding a matched control set, or at minimum reporting accuracy on CF items whose gold answer is identical to their GF counterpart, before interpreting the Activation/Deactivation asymmetry as directional difficulty in withdrawing warnings.
- [§4.4] The claim that models 'acknowledge' the changed patient information rests on the statistic that 85.6% of incorrect deactivation responses mention the changed information. Mentioning an attribute is not the same as correctly reasoning about it, and the same mention rate is not reported for correct deactivation responses or for GF responses, so the 85.6% figure has no baseline. The representative Lingshu-32B case is persuasive, but a systematic coding of whether the rationale contains a rule-consistent use of the changed information would strengthen the recognition-versus-application distinction.
minor comments (5)
- [§3.2] The phrase 'whenever possible' should be replaced with an explicit operational rule and a release-time audit of which non-target clinical details changed across each counterfactual pair.
- [Table 2] The column headers 'Overall↑GF↑CF↑...' are run together in the rendered table, and the table would be easier to read if model family and parameter count were listed in a separate column.
- [§4.1] GPT-5.2 is cited to the same reference as GPT-5; if it is a distinct model, a separate or supplemental citation is needed.
- [Figure 4(c)] The legend and the black diamonds for means could be clarified, and the statement that gestational-week thresholds occur only in GF questions should appear in the main text rather than only in the caption.
- [Ethical Statement] The Ethical Statement appropriately limits clinical interpretation, but the main text would benefit from a limitations paragraph noting that the benchmark intentionally excludes uncertainty, patient preferences, and local guidance, which are material to real prescribing decisions.
Circularity Check
No significant circularity: the benchmark's ground truth is externally anchored and no quantity is fitted to the evaluated models.
full rationale
The paper's central claim is an empirically measured accuracy gap on a benchmark whose gold answers are fixed by external clinical sources (2023 AGS Beers Criteria, FDA PLLR, TGA Prescribing Medicines in Pregnancy Database, 2025 KIDs List) plus expert validation. Nothing in the pipeline fits a parameter to the evaluated models, and no model output feeds back into the ground truth. The GF–CF comparison, the activation–deactivation asymmetry, and the 69.8% retention statistic are all computed against these externally fixed labels. The construction pipeline (MedRule2Pair) transforms source recommendations into rules and then into questions; the correctness of that transformation is a validity question, not a circularity, because the answer sets are derived from the source rules rather than from model behavior. The potential confounds raised by the skeptic—unmatched GF and CF question pools, the absence of a difficulty-matched control where patient information changes but the gold answer stays fixed, and possible format effects from 'None of the above' options—are threats to internal validity of the headline gap, but they do not make any result equivalent to its own inputs by construction. There is also no load-bearing self-citation: the evaluated models are external, and the authors' own affiliated model Lingshu-32B is highlighted as a failure example rather than used to justify the conclusion. Accordingly, no circular step can be exhibited, and the honest finding is no significant circularity.
Assumptions & free parameters
assumptions (4)
- domain assumption The four source compilations (2023 AGS Beers Criteria, FDA PLLR, TGA Prescribing Medicines in Pregnancy Database, 2025 KIDs List) are authoritative and their recommendations are correctly encoded as rules.
- domain assumption Expert validation reliably establishes ground-truth correctness and counterfactual consistency for all 467 questions.
- ad hoc to paper The deterministic generation scripts produce CF variants that differ from GF variants only in the targeted patient information, with all other details fixed.
- domain assumption Zero-shot exact-match accuracy on the multiple-choice format adequately measures the intended medication-safety reasoning ability.
Cite this review
Pith. "Pith review of Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning." pith.science (2026). https://pith.science/paper/DNCMWRCP
@misc{pith2026260803028,
author = {Pith},
title = {Pith review of: Evaluating Counterfactual Sensitivity to Patient Information in Medication-Safety Reasoning},
year = {2026},
howpublished = {\url{https://pith.science/paper/DNCMWRCP}},
note = {Machine review of arXiv:2608.03028}
}
read the original abstract
Applying a valid medication-safety rule when its patient-specific conditions are not met can produce an incorrect decision. Existing medical evaluations largely use isolated and fixed scenarios. A model may therefore answer correctly by recalling a drug-risk association without showing that it used patient information to decide whether the rule applies. To address this gap, we introduce MedPIC-Bench, a benchmark of source-verifiable recommendations and expert-validated questions for patient-specific medication-safety reasoning. It combines guideline-following questions with paired counterfactual questions in which a controlled change in patient information changes whether a rule applies. The benchmark contains 467 questions annotated along six clinical and reasoning dimensions. Across 28 medical-specific, general, and proprietary LLMs, every model performs worse on counterfactual questions, with mean accuracy falling from 63.6\% to 45.1\%. Models perform well when an explicit patient attribute directly signals a familiar contraindication, but struggle when patient information must narrow or withdraw a safety warning. Model rationales often acknowledge the changed patient information, yet the final answers retain the previous safety judgment. This vulnerability persists among medical-specific LLMs, whose average CF performance trails that of general LLMs. MedPIC-Bench therefore makes conditional rule application measurable and highlights the limitations of static medication-safety accuracy for assessing patient-specific reliability.
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
-
[1]
Nature , volume=
Large language models encode clinical knowledge , author=. Nature , volume=. 2023 , publisher=
2023
-
[2]
Nature medicine , volume=
Toward expert-level medical question answering with large language models , author=. Nature medicine , volume=. 2025 , publisher=
2025
-
[3]
arXiv preprint arXiv:2505.08775 , year=
Healthbench: Evaluating large language models towards improved human health , author=. arXiv preprint arXiv:2505.08775 , year=
-
[4]
Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence , pages=
MultifacetEval: multifaceted evaluation to probe LLMs in mastering medical knowledge , author=. Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence , pages=
-
[5]
arXiv preprint arXiv:2505.11613 , year=
Medguide: Benchmarking clinical decision-making in large language models , author=. arXiv preprint arXiv:2505.11613 , year=
-
[6]
arXiv preprint arXiv:2605.26567 , year=
MedGuideX: Internalizing Decision Logic from Executable Guidelines into Large Language Models for Clinical Reasoning , author=. arXiv preprint arXiv:2605.26567 , year=
-
[7]
Proceedings of the AAAI Conference on Artificial Intelligence , volume=
CounterBench: Evaluating and improving counterfactual reasoning in large language models , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=
-
[8]
Rx-LLM: a benchmarking suite to evaluate safe large language model performance for medication-related tasks , author=. medRxiv , pages=. 2025 , publisher=
work page 2025
Show all 34 references
-
[9]
2025 IEEE International Conference on Bioinformatics and Biomedicine (BIBM) , pages=
RxSafeBench: Identifying Medication Safety Issues of Large Language Models in Simulated Consultation , author=. 2025 IEEE International Conference on Bioinformatics and Biomedicine (BIBM) , pages=. 2025 , organization=
2025
-
[10]
arXiv preprint arXiv:2605.14543 , year=
RxEval: A Prescription-Level Benchmark for Evaluating LLM Medication Recommendation , author=. arXiv preprint arXiv:2605.14543 , year=
-
[11]
How Can We Diagnose and Treat Bias in Large Language Models for Clinical Decision-Making? , author=. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) ...
2025
-
[12]
2025 , publisher=
Ghosh, Rajarshi and Gupta, Abhay and McBride, Hudson and Vaidya, Anurag Jayant and Mahmood, Faisal , booktitle=. 2025 , publisher=
2025
-
[13]
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=
MedEinst: Benchmarking the Einstellung Effect in Medical LLMs through Counterfactual Differential Diagnosis , author=. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=
-
[14]
Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=
Qu, Zhan and F. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=
-
[15]
medRxiv , pages=
A Three-Tier Operational Benchmark for Evaluating Large Language Models on Hospital Medication Safety , author=. medRxiv , pages=. 2026 , publisher=
2026
-
[16]
Journal of the American Geriatrics Society , volume=
American Geriatrics Society 2023 updated AGS Beers Criteria. Journal of the American Geriatrics Society , volume=. 2023 , publisher=
2023
-
[17]
The Journal of Pediatric Pharmacology and Therapeutics , volume=
Pediatric Pharmacy Association 2025 KIDs List of key potentially inappropriate drugs in pediatrics , author=. The Journal of Pediatric Pharmacology and Therapeutics , volume=. 2025 , publisher=
2025
-
[18]
Advances in neural information processing systems , volume=
Pytorch: An imperative style, high-performance deep learning library , author=. Advances in neural information processing systems , volume=
-
[19]
Proceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations , pages=
Transformers: State-of-the-art natural language processing , author=. Proceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations , pages=
2020
-
[20]
Advances in neural information processing systems , volume=
Sglang: Efficient execution of structured language model programs , author=. Advances in neural information processing systems , volume=
-
[21]
International Conference on Machine Learning , pages=
HealthGPT: A Medical Large Vision-Language Model for Unifying Comprehension and Generation via Heterogeneous Knowledge Adaptation , author=. International Conference on Machine Learning , pages=. 2025 , organization=
2025
-
[22]
arXiv preprint arXiv:2507.05201 , year=
Medgemma technical report , author=. arXiv preprint arXiv:2507.05201 , year=
-
[23]
arXiv preprint arXiv:2506.07044 , year=
Lingshu: A generalist foundation model for unified multimodal medical understanding and reasoning , author=. arXiv preprint arXiv:2506.07044 , year=
-
[24]
arXiv preprint arXiv:2510.08668 , year=
Hulu-med: A transparent generalist model towards holistic medical vision-language understanding , author=. arXiv preprint arXiv:2510.08668 , year=
-
[25]
Findings of the association for computational linguistics: EMNLP 2023 , pages=
Huatuogpt, towards taming language model to be a doctor , author=. Findings of the association for computational linguistics: EMNLP 2023 , pages=
2023
-
[26]
Findings of the Association for Computational Linguistics: ACL 2026 , pages=
Scaling is Not All You Need: Clinical-Oriented Reinforcement Learning Makes Parameter-Efficient Clinical Reasoning , author=. Findings of the Association for Computational Linguistics: ACL 2026 , pages=
2026
-
[27]
arXiv preprint arXiv:2509.02208 , year=
Baichuan-m2: Scaling medical capability with large verifier system , author=. arXiv preprint arXiv:2509.02208 , year=
-
[28]
2025 , eprint=
Gemma 3 Technical Report , author=. 2025 , eprint=
2025
-
[29]
Neural Information Processing Systems , year=
The Llama 3 herd of models , author=. Neural Information Processing Systems , year=
-
[30]
arXiv preprint arXiv:2508.10925 , year=
gpt-oss-120b & gpt-oss-20b model card , author=. arXiv preprint arXiv:2508.10925 , year=
-
[31]
5-omni technical report , author=
Qwen3. 5-omni technical report , author=. arXiv preprint arXiv:2604.15804 , year=
-
[32]
arXiv preprint arXiv:2507.06261 , year=
Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities , author=. arXiv preprint arXiv:2507.06261 , year=
-
[33]
arXiv preprint arXiv:2601.03267 , year=
Openai gpt-5 system card , author=. arXiv preprint arXiv:2601.03267 , year=
-
[34]
arXiv preprint arXiv:2606.19348 , year=
Deepseek-v4: Towards highly efficient million-token context intelligence , author=. arXiv preprint arXiv:2606.19348 , year=
Reviewed August 8, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.