REVIEW 6 major objections 6 minor 1 cited by
Standard medical benchmarks hide a systematic fragility in clinical LLMs: base accuracy overstates robust accuracy by 16–28 percentage points, and the best counterfactual-tested system still fails one in five altered presentations.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-02 02:13 UTC pith:QAJH2MII
load-bearing objection New benchmark, sensible retrieval fix, but headline gap numbers are not credible as stated: internal inconsistency, unvalidated judge, no uncertainty, scaffold tuned on test set. the 6 major comments →
MamaBench: Benchmarking LLM Robustness in Maternal and Child Health Diagnosis through Counterfactual Clinical Perturbation
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
On the paper's own terms, the discovery is that LLMs exhibit diagnostic fixation: they get the base narrative right, then rigidly repeat the same diagnosis when a single discriminative parameter shifts the ground truth. This failure is quantified by the Bias Trap Rate (BTR), the conditional probability of failing the counterfactual given success on the base. The paper reports BTR between 20.3% and 50.4% across eight configurations, with base accuracy overstating robust accuracy by 16.2 to 27.6 percentage points. It further claims that standard retrieval-augmented generation leaves BTR unchanged because base and counterfactual retrieve near-identical contexts, whereas the proposed evidence-an
What carries the argument
The central object is the Bias Trap Rate (BTR), a conditional probability that turns diagnostic fixation into a number: the share of base-case successes that flip to failure when the narrative is minimally perturbed. The paper's proposed mechanism is Evidence-Anchored RAG (EA-RAG), an inference-time pipeline with three stages: typed clinical parameter extraction, coverage auditing that finds which extracted parameters are missing from retrieved chunks, and contrastive sub-queries that fill those gaps, followed by a taxonomy-grounded generation scaffold. The taxonomy comes from a failure-mode analysis of 113 baseline errors, with the most common being missed critical info (48.7%) and underthi
Load-bearing premise
The entire set of numeric claims rests on the assumption that the LLM-as-judge (Claude Opus 4.5) correctly decides whether a model's diagnosis matches the reference; if the judge's blind spots line up with particular diagnoses or perturbation categories, the reported BTR and robust-accuracy values are systematically off.
What would settle it
Take a sample of MamaBench model outputs (e.g., 50 pairs across several models), have two or more independent clinicians, blinded to the judge's labels, grade whether each output matches the ground-truth diagnosis, and measure agreement between the clinicians and the LLM judge. If agreement is poor (e.g., Cohen's kappa below 0.8) or disagreements cluster in certain perturbation categories, the paper's central numbers lose their grounding.
If this is right
- If the central claim holds, a model can look reliable on benchmark questions while failing to distinguish clinically similar presentations, so deployment decisions in maternal or pediatric care should be based on robust accuracy, not base accuracy.
- Standard top-k retrieval is not a remedy: because base and counterfactual narratives embed almost identically, vanilla RAG retrieves the same context and preserves the failure.
- Retrieval-level interventions alone cannot solve the problem: the best EA-RAG configuration still yields a 20.3% BTR, meaning a system that appears competent would miss a management-changing detail in about one in five counterfactual cases.
- Evidence-anchored retrieval gives larger gains on stronger models, suggesting that once reasoning capacity is sufficient, retrieval quality becomes the bottleneck; weaker models are limited by reasoning rather than context.
- The ablation indicates the generation scaffold contributes more than coverage-aware retrieval alone, and the two mechanisms are complementary: the scaffold changes how the model reasons, while coverage retrieval changes what evidence it sees.
Where Pith is reading between the lines
- Editorial extension: the BTR metric is not inherently medical; it could be applied to any domain where a minimal edit changes the correct decision, such as legal or financial advice, to expose similar fixation.
- Editorial extension: because all scores come from one LLM judge with no clinician adjudication, the reported gaps assume the judge's errors do not correlate with perturbation type; if they do, the true gaps could be larger or smaller. A clinician-annotated subset would settle this.
- Editorial extension: the counterfactual pairs are English-only and maternal/pediatric; extending MamaBench to other languages or specialties could reveal whether the robustness gap is a general LLM property or specific to this clinical domain's language and case distribution.
- Editorial extension: EA-RAG operates at inference time, so it could be combined with training-time interventions such as counterfactual fine-tuning to test whether the residual 20% BTR can be pushed below the paper's current floor.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces MamaBench, a counterfactual benchmark for maternal and paediatric LLM diagnosis, consisting of 217 expert-authored base/counterfactual pairs across 371 pathologies. It defines Diagnostic Fixation and uses the Bias Trap Rate (BTR) as the primary metric. The authors also propose Evidence-Anchored RAG (EA-RAG), a three-stage inference-time retrieval method with evidence extraction, coverage auditing, and a taxonomy-grounded scaffold. Evaluations across eight configurations of several LLMs lead to the central claims: base accuracy overstates robust accuracy by 16–28 percentage points in every model; vanilla RAG provides no counterfactual benefit; and EA-RAG reduces BTR by 5.5 pp on Claude Sonnet 4.6 without degrading base accuracy, although a residual ~20% BTR remains.
Significance. If the results hold, the paper addresses a real and important gap: standard medical QA benchmarks do not measure whether a model can distinguish clinically similar presentations requiring different management. The expert-authored counterfactual-pair design is a substantive contribution, and the BTR formulation directly quantifies conditional fragility. The data-separation decision (excluding benchmark cases from the retrieval corpus) is a good experimental practice. The paper also makes a practical intervention claim for EA-RAG at inference time. These contributions are potentially valuable for clinical LLM evaluation, but the empirical conclusions and the EA-RAG advantage rest on several validation and statistical issues that need to be resolved before the claims are reliable.
major comments (6)
- [Abstract; §5.2, Table 1] The headline range “16–28 percentage points” is contradicted by the paper’s own Table 1. The robustness gap (Acc_base − Acc_rob) is 30.4 pp for Kimi K2.5 (76.0–45.6) and 30.9 pp for GPT-4o RAG (77.4–46.5). Additionally, §5.2 states the minimum is 16.2 pp for Claude + EA-RAG (82.0→65.0), but Table 1 lists Acc_base = 81.6, giving 16.6 pp. The abstract and §5.2 must be corrected to match the table, and any claim about the range must be recomputed from the reported data.
- [§3.3; Limitations] All headline metrics—Acc_base, Acc_rob, and especially BTR—are produced by a single LLM judge (Claude Opus 4.5, T=0) with no clinician-adjudicated sample, inter-annotator agreement, or error analysis. The judge is instructed to be “strict on clinically significant modifiers that change management,” which is exactly the dimension along which the counterfactual pairs are constructed (§3.2). This design can systematically inflate BTR if the judge penalizes a correct diagnosis that does not explicitly address a changed management requirement. The Limitations section concedes this risk, but because BTR is the paper’s primary metric and the robustness gap is the core finding, the authors should provide a validation subset scored by clinicians, a judge error analysis, or at minimum an independent agreement study. This is load-bearing, not optional.
- [§4.3; §5.3] EA-RAG’s scaffold was derived from an error taxonomy of 113 baseline failures on MamaBench, and the thresholds θ=0.6, δ=0.82, k=5, and k+3 are hand-set on the same benchmark. The method is then evaluated on the same 217 pairs. This is a selection-on-the-test-set risk: the reported EA-RAG improvements (especially the 5.5 pp BTR reduction) may be optimistic and may not generalize to new counterfactual pairs. The authors should report a held-out development split, cross-validation, or at least a clear separation between the set used for failure taxonomy/threshold tuning and the final evaluation set. Without this, the EA-RAG gain is not yet convincingly established.
- [§5.2; Table 1] The claim that “Vanilla RAG does not help” is not supported by the reported experiments. The only GPT-4o configuration is the RAG baseline (Table 1); no standalone GPT-4o result is reported. Comparing GPT-4o+RAG to Kimi K2.5 standalone does not isolate the effect of retrieval. The authors should add a standalone GPT-4o condition or otherwise directly compare matched models with and without RAG before concluding that RAG provides no counterfactual benefit.
- [§5.3, Table 2] The ablative interpretation is internally inconsistent with the pair counts. The text states “Seven additional pairs shift from PF → PP (and seven from FF → FP/PP)” when comparing k5 to k5_scaffold, but the table shows PP 94→101 (+7) while PF actually increases 61→62, not decreases; FF drops 26→19. The counts cannot support seven PF→PP transitions. Please correct the transition description and provide a proper transition matrix or a revised interpretation.
- [§5.1, Table 1, Table 2] All metrics are point estimates over N=217 pairs with no confidence intervals, bootstrap, or significance tests. BTR numerators are small (e.g., PF≈36 for Claude EA-RAG, PF≈61 for GPT-4o), so a 4–5.5 pp BTR difference may be within sampling noise. For example, approximate 95% Wilson intervals on BTR should be reported. The authors should also report the raw PP/PF/FP/FF counts for Table 1, not only percentages, so that readers can assess precision.
minor comments (6)
- [§1, §5.1] The paper describes “four frontier LLMs” but evaluates six models (GLM-5, Kimi K2.5, Nemotron 120B, GPT-4o, GPT-5.4, Claude Sonnet 4.6) from at least five families. Please correct the count.
- [Figure 2] The y-axis label “Robustness” is vague; it should say “Acc_rob” and the x-axis “BTR” is already fine. Also clarify whether the standalone markers include GPT-4o RAG or not.
- [§4.3] There is a missing period or conjunction in “Self-RAG (Asai et al., 2024) EA-RAG operates entirely at inference time.” Please rephrase.
- [References] The reference for Li et al. (2025) appears after Zheng et al. (2023), breaking the alphabetical order. Also consider adding a reference for the embedding model (text-embedding-3-large) and for the GPT-5.4-nano extractor, if these are public artifacts.
- [§3.3] The definition of Acc_rob = |PP|/N is fine, but the names “Acc_base” and “Acc_rob” should be introduced with explicit equations. Currently BTR has an equation but the other two do not.
- [§4.2] The coverage audit uses cos(ϕ(e_j), ϕ(c_i)) but the embedding dimension is stated for the query side; please clarify that the same encoder is used for both evidence elements and chunks.
Circularity Check
No equation-level circularity in the benchmark metrics; the central robustness-gap finding is measured, not derived. The only partial circularity is in-sample fitting of EA-RAG's scaffold and thresholds to the same benchmark used for the reported BTR gains.
specific steps
-
fitted input called prediction
[§4.3 (Stage 3) and §5.2 (Main Results)]
"The generator receives the narrative, Rfinal, and a scaffold instruction set derived from error taxonomy analysis of 113 baseline failures: ... Missed critical info 48.7% ... EA-RAG achieves 20.3% BTR and 65.0% robust accuracy on Claude Sonnet 4.6, a 5.5 percentage point BTR reduction without degrading base accuracy."
The scaffold instructions are fitted to the error distribution of the same MamaBench cases on which EA-RAG's BTR reduction is then reported, with θ=0.6 and δ=0.82 hand-set and no held-out split described. The instructions directly dictate the behavior counted by BTR (e.g., 'Analyse ALL clinical parameters' for the dominant 'Missed critical info' failure), so the measured improvement is not an independent prediction; it is partly in-sample. This is partial circularity/overfitting, not a derivation-level identity, and the headline robustness-gap finding does not depend on it.
full rationale
The core claims—Acc_base, Acc_rob, and BTR—are measured conditional frequencies defined by Eq. (2), not quantities derived from fitted parameters; no equation in the paper reduces to its inputs. BTR is adopted from external work (Chen et al., 2026), with no author overlap, so there is no self-citation chain. 'Diagnostic Fixation' is a definition, not a predicted result. The main circularity-adjacent issue is that EA-RAG's scaffold was constructed from an error taxonomy of 113 baseline failures on MamaBench and evaluated on the same benchmark (with hand-set θ, δ and no held-out split), making the reported 5.5 pp BTR gain partly in-sample. The manuscript's own Limitations section concedes the separate LLM-as-judge validity risk, which could inflate BTR, but that is a measurement-validity concern rather than a circular derivation. On balance: central empirical gap is independent and self-contained; the method-evaluation component merits a moderate circularity/overfitting flag, so score 4.
Axiom & Free-Parameter Ledger
free parameters (3)
- coverage threshold θ =
0.6
- deduplication threshold δ =
0.82
- retrieval count k and cap k+3 =
k=5, cap=8
axioms (5)
- domain assumption LLM-as-judge (Claude Opus 4.5) correctly classifies open-ended model outputs as clinically correct or wrong.
- domain assumption Expert-authored base and counterfactual narratives have valid, unambiguous single ground-truth diagnoses.
- domain assumption The retrieval corpus contains no MamaBench case narratives or labels.
- domain assumption Cosine similarity on text-embedding-3-large embeddings captures clinically meaningful parameter coverage.
- ad hoc to paper The scaffold derived from 113 baseline failures generalizes to all MamaBench pairs.
read the original abstract
Large language models achieve strong scores on medical benchmarks, yet these benchmarks evaluate each question in isolation, providing no measure of whether a system can distinguish clinically similar presentations requiring different interventions. We introduce MamaBench, the first counterfactual benchmark for maternal and paediatric AI: 434 expert-authored clinical narratives in 217 pairs across 371 pathologies, evaluated via the Bias Trap Rate (BTR), the conditional probability that a model fails the counterfactual given success on the base case. We propose Evidence-Anchored RAG (EA-RAG), a three-stage retrieval method that replaces aggregate similarity with an evidence coverage objective through clinical parameter extraction, coverage auditing, and contrastive sub-queries. Across eight configurations of four frontier LLMs, base accuracy overstates robust accuracy by 16-28 percentage points in every model. EA-RAG achieves 20.3% BTR and 65.0% robust accuracy on Claude Sonnet 4.6, a 5.5 percentage point BTR reduction without degrading base accuracy. The residual 20% BTR confirms that counterfactual robustness in clinical AI remains an open challenge. Keywords: counterfactual evaluation, clinical AI, maternal healthcare, retrieval-augmented generation, diagnostic robustness
Figures
Forward citations
Cited by 1 Pith paper
-
IyawoBench v2.0: Extended Diagnostic Evaluation of Large Language Model Clinical Triage in Nigerian Primary Care
A new triage benchmark reports that binary safety metrics hide a 77-point under-triage gap for Llama 3.1 8B and that the best LLM changes by deployment scenario, though the paper's formal failure-mode labels are inter...
Reference graph
Works this paper leans on
-
[1]
Asai, A., Wu, Z., Wang, Y., Sil, A., & Hajishirzi, H. (2024). Self- RAG : Learning to retrieve, generate, and critique through self-reflection. ICLR
2024
-
[2]
Chen, W., Huang, G., Wang, W., & Zhu, Z. (2026). MedEinst : Benchmarking the E instellung Effect in medical LLM s through counterfactual differential diagnosis. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026)
2026
-
[3]
Fansi Tchango, A., Goel, R., Wen, Z., Martel, J., & Ghosn, J. (2022). DDXPlus : A new dataset for automatic medical diagnosis. NeurIPS
2022
-
[4]
Gardner, M., Artzi, Y., Basmova, V., et al. (2020). Evaluating models' local decision boundaries via contrast sets. Findings of EMNLP, 1307--1323
2020
-
[5]
F., Gao, L., et al
Jiang, Z., Xu, F. F., Gao, L., et al. (2023). Active retrieval augmented generation. EMNLP, 7969--7992
2023
-
[6]
Jin, D., Pan, E., Oufattole, N., et al. (2021). What disease does this patient have? Applied Sciences, 11(14), 6421
2021
-
[7]
W., & Lu, X
Jin, Q., Dhingra, B., Liu, Z., Cohen, W. W., & Lu, X. (2019). PubMedQA : A dataset for biomedical research question answering. EMNLP, 2567--2577
2019
-
[8]
Karpukhin, V., Oguz, B., Min, S., et al. (2020). Dense passage retrieval for open-domain question answering. EMNLP, 6769--6781
2020
-
[9]
Kaushik, D., Hovy, E., & Lipton, Z. C. (2020). Learning the difference that makes a difference with counterfactually-augmented data. ICLR
2020
-
[10]
Khattab, O., Santhanam, K., Li, X. L., et al. (2023). Demonstrate- S earch- P redict: Composing retrieval and language models for knowledge-intensive NLP . arXiv:2212.14024
Pith/arXiv arXiv 2023
-
[11]
Lewis, P., Perez, E., Piktus, A., et al. (2020). Retrieval-augmented generation for knowledge-intensive NLP tasks. NeurIPS, 9459--9474
2020
-
[12]
Luchins, A. S. (1942). Mechanization in problem solving: The effect of E instellung. Psych.\ Monographs, 54(6), i--95
1942
-
[13]
Nori, H., King, N., McKinney, S. M., et al. (2023). Capabilities of GPT-4 on medical competency examinations. arXiv:2303.13375
Pith/arXiv arXiv 2023
-
[14]
K., & Sankarasubbu, M
Pal, A., Umapathi, L. K., & Sankarasubbu, M. (2022). MedMCQA : A large-scale multi-subject multi-choice dataset for medical domain question answering. CHIL, 248--260
2022
-
[15]
R., Cole-Lewis, H., Sayres, R., et al
Pfohl, S. R., Cole-Lewis, H., Sayres, R., et al. (2024). A toolbox for surfacing health equity harms and biases in large language models. arXiv:2403.12025
Pith/arXiv arXiv 2024
-
[16]
T., Wu, T., Guestrin, C., & Singh, S
Ribeiro, M. T., Wu, T., Guestrin, C., & Singh, S. (2020). Beyond accuracy: Behavioral testing of NLP models with C heck L ist. ACL, 4902--4912
2020
-
[17]
Shi, F., Chen, X., Misra, K., et al. (2023). Large language models can be easily distracted by irrelevant context. ICML
2023
-
[18]
Singhal, K., Azizi, S., Tu, T., et al. (2023). Large language models encode clinical knowledge. Nature, 620, 172--180
2023
-
[19]
Topol, E. J. (2019). High-performance medicine: The convergence of human and artificial intelligence. Nature Med., 25, 44--56
2019
-
[20]
Trivedi, H., Balasubramanian, N., Khot, T., & Sabharwal, A. (2023). Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions. ACL
2023
-
[21]
Wang, L., Yang, N., & Wei, F. (2023). Q uery2doc: Query expansion with large language models. EMNLP, 9414--9423
2023
-
[22]
Wei, J., Wang, X., Schuurmans, D., et al. (2022). Chain-of-thought prompting elicits reasoning in large language models. NeurIPS, 24824--24837
2022
-
[23]
Yao, S., Yu, D., Zhao, J., et al. (2023). Tree of thoughts: Deliberate problem solving with large language models. NeurIPS
2023
-
[24]
Zakka, C., Shad, R., Chaurasia, A., et al. (2024). Almanac: Retrieval-augmented language models for clinical medicine. NEJM AI, 1(2)
2024
-
[25]
Zheng, L., Chiang, W.-L., Sheng, Y., et al. (2023). Judging LLM -as-a-judge with MT-Bench and C hatbot A rena. NeurIPS
2023
-
[26]
Li, M., Zhan, Z., Yang, H., Xiao, Y., Zhou, H., Huang, J., & Zhang, R. (2025). Benchmarking retrieval-augmented large language models in biomedical NLP : Application, robustness, and self-awareness. Science Advances, 11(47), eadr1443
2025
-
[27]
Zhan, Z., Wang, J., Zhou, S., Deng, J., & Zhang, R. (2025). MMRAG : multi-mode retrieval-augmented generation with large language models for biomedical in-context learning. Journal of the American Medical Informatics Association, 32(10), 1505--1516
2025
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.