Pith. sign in

REVIEW 6 major objections 6 minor 1 cited by

Standard medical benchmarks hide a systematic fragility in clinical LLMs: base accuracy overstates robust accuracy by 16–28 percentage points, and the best counterfactual-tested system still fails one in five altered presentations.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

T0 review · deepseek-v4-flash

2026-08-02 02:13 UTC pith:QAJH2MII

load-bearing objection New benchmark, sensible retrieval fix, but headline gap numbers are not credible as stated: internal inconsistency, unvalidated judge, no uncertainty, scaffold tuned on test set. the 6 major comments →

arxiv 2607.14385 v1 pith:QAJH2MII submitted 2026-07-15 cs.CL cs.LG

MamaBench: Benchmarking LLM Robustness in Maternal and Child Health Diagnosis through Counterfactual Clinical Perturbation

classification cs.CL cs.LG
keywords counterfactual evaluationclinical AImaternal healthcarepediatricdiagnostic robustnessbias trap rateretrieval-augmented generationLLM-as-judge
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that standard accuracy scores on medical benchmarks give a false sense of safety for LLMs in maternal and child health. It introduces MamaBench, 434 expert-written patient narratives arranged as 217 counterfactual pairs that differ by a minimal clinical detail but have different correct diagnoses. Across every model tested, base accuracy overstates robust accuracy by 16 to 28 percentage points: a model can answer the base case correctly yet fail the near-identical twin case. The paper's retrieval method, EA-RAG, narrows the gap by up to 5.5 points without lowering base accuracy, but the best configuration still trips on one in five pairs. The message is that counterfactual robustness should be measured before clinical deployment, not assumed from exam-style benchmarks.

Core claim

On the paper's own terms, the discovery is that LLMs exhibit diagnostic fixation: they get the base narrative right, then rigidly repeat the same diagnosis when a single discriminative parameter shifts the ground truth. This failure is quantified by the Bias Trap Rate (BTR), the conditional probability of failing the counterfactual given success on the base. The paper reports BTR between 20.3% and 50.4% across eight configurations, with base accuracy overstating robust accuracy by 16.2 to 27.6 percentage points. It further claims that standard retrieval-augmented generation leaves BTR unchanged because base and counterfactual retrieve near-identical contexts, whereas the proposed evidence-an

What carries the argument

The central object is the Bias Trap Rate (BTR), a conditional probability that turns diagnostic fixation into a number: the share of base-case successes that flip to failure when the narrative is minimally perturbed. The paper's proposed mechanism is Evidence-Anchored RAG (EA-RAG), an inference-time pipeline with three stages: typed clinical parameter extraction, coverage auditing that finds which extracted parameters are missing from retrieved chunks, and contrastive sub-queries that fill those gaps, followed by a taxonomy-grounded generation scaffold. The taxonomy comes from a failure-mode analysis of 113 baseline errors, with the most common being missed critical info (48.7%) and underthi

Load-bearing premise

The entire set of numeric claims rests on the assumption that the LLM-as-judge (Claude Opus 4.5) correctly decides whether a model's diagnosis matches the reference; if the judge's blind spots line up with particular diagnoses or perturbation categories, the reported BTR and robust-accuracy values are systematically off.

What would settle it

Take a sample of MamaBench model outputs (e.g., 50 pairs across several models), have two or more independent clinicians, blinded to the judge's labels, grade whether each output matches the ground-truth diagnosis, and measure agreement between the clinicians and the LLM judge. If agreement is poor (e.g., Cohen's kappa below 0.8) or disagreements cluster in certain perturbation categories, the paper's central numbers lose their grounding.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the central claim holds, a model can look reliable on benchmark questions while failing to distinguish clinically similar presentations, so deployment decisions in maternal or pediatric care should be based on robust accuracy, not base accuracy.
  • Standard top-k retrieval is not a remedy: because base and counterfactual narratives embed almost identically, vanilla RAG retrieves the same context and preserves the failure.
  • Retrieval-level interventions alone cannot solve the problem: the best EA-RAG configuration still yields a 20.3% BTR, meaning a system that appears competent would miss a management-changing detail in about one in five counterfactual cases.
  • Evidence-anchored retrieval gives larger gains on stronger models, suggesting that once reasoning capacity is sufficient, retrieval quality becomes the bottleneck; weaker models are limited by reasoning rather than context.
  • The ablation indicates the generation scaffold contributes more than coverage-aware retrieval alone, and the two mechanisms are complementary: the scaffold changes how the model reasons, while coverage retrieval changes what evidence it sees.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: the BTR metric is not inherently medical; it could be applied to any domain where a minimal edit changes the correct decision, such as legal or financial advice, to expose similar fixation.
  • Editorial extension: because all scores come from one LLM judge with no clinician adjudication, the reported gaps assume the judge's errors do not correlate with perturbation type; if they do, the true gaps could be larger or smaller. A clinician-annotated subset would settle this.
  • Editorial extension: the counterfactual pairs are English-only and maternal/pediatric; extending MamaBench to other languages or specialties could reveal whether the robustness gap is a general LLM property or specific to this clinical domain's language and case distribution.
  • Editorial extension: EA-RAG operates at inference time, so it could be combined with training-time interventions such as counterfactual fine-tuning to test whether the residual 20% BTR can be pushed below the paper's current floor.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

6 major / 6 minor

Summary. The paper introduces MamaBench, a counterfactual benchmark for maternal and paediatric LLM diagnosis, consisting of 217 expert-authored base/counterfactual pairs across 371 pathologies. It defines Diagnostic Fixation and uses the Bias Trap Rate (BTR) as the primary metric. The authors also propose Evidence-Anchored RAG (EA-RAG), a three-stage inference-time retrieval method with evidence extraction, coverage auditing, and a taxonomy-grounded scaffold. Evaluations across eight configurations of several LLMs lead to the central claims: base accuracy overstates robust accuracy by 16–28 percentage points in every model; vanilla RAG provides no counterfactual benefit; and EA-RAG reduces BTR by 5.5 pp on Claude Sonnet 4.6 without degrading base accuracy, although a residual ~20% BTR remains.

Significance. If the results hold, the paper addresses a real and important gap: standard medical QA benchmarks do not measure whether a model can distinguish clinically similar presentations requiring different management. The expert-authored counterfactual-pair design is a substantive contribution, and the BTR formulation directly quantifies conditional fragility. The data-separation decision (excluding benchmark cases from the retrieval corpus) is a good experimental practice. The paper also makes a practical intervention claim for EA-RAG at inference time. These contributions are potentially valuable for clinical LLM evaluation, but the empirical conclusions and the EA-RAG advantage rest on several validation and statistical issues that need to be resolved before the claims are reliable.

major comments (6)
  1. [Abstract; §5.2, Table 1] The headline range “16–28 percentage points” is contradicted by the paper’s own Table 1. The robustness gap (Acc_base − Acc_rob) is 30.4 pp for Kimi K2.5 (76.0–45.6) and 30.9 pp for GPT-4o RAG (77.4–46.5). Additionally, §5.2 states the minimum is 16.2 pp for Claude + EA-RAG (82.0→65.0), but Table 1 lists Acc_base = 81.6, giving 16.6 pp. The abstract and §5.2 must be corrected to match the table, and any claim about the range must be recomputed from the reported data.
  2. [§3.3; Limitations] All headline metrics—Acc_base, Acc_rob, and especially BTR—are produced by a single LLM judge (Claude Opus 4.5, T=0) with no clinician-adjudicated sample, inter-annotator agreement, or error analysis. The judge is instructed to be “strict on clinically significant modifiers that change management,” which is exactly the dimension along which the counterfactual pairs are constructed (§3.2). This design can systematically inflate BTR if the judge penalizes a correct diagnosis that does not explicitly address a changed management requirement. The Limitations section concedes this risk, but because BTR is the paper’s primary metric and the robustness gap is the core finding, the authors should provide a validation subset scored by clinicians, a judge error analysis, or at minimum an independent agreement study. This is load-bearing, not optional.
  3. [§4.3; §5.3] EA-RAG’s scaffold was derived from an error taxonomy of 113 baseline failures on MamaBench, and the thresholds θ=0.6, δ=0.82, k=5, and k+3 are hand-set on the same benchmark. The method is then evaluated on the same 217 pairs. This is a selection-on-the-test-set risk: the reported EA-RAG improvements (especially the 5.5 pp BTR reduction) may be optimistic and may not generalize to new counterfactual pairs. The authors should report a held-out development split, cross-validation, or at least a clear separation between the set used for failure taxonomy/threshold tuning and the final evaluation set. Without this, the EA-RAG gain is not yet convincingly established.
  4. [§5.2; Table 1] The claim that “Vanilla RAG does not help” is not supported by the reported experiments. The only GPT-4o configuration is the RAG baseline (Table 1); no standalone GPT-4o result is reported. Comparing GPT-4o+RAG to Kimi K2.5 standalone does not isolate the effect of retrieval. The authors should add a standalone GPT-4o condition or otherwise directly compare matched models with and without RAG before concluding that RAG provides no counterfactual benefit.
  5. [§5.3, Table 2] The ablative interpretation is internally inconsistent with the pair counts. The text states “Seven additional pairs shift from PF → PP (and seven from FF → FP/PP)” when comparing k5 to k5_scaffold, but the table shows PP 94→101 (+7) while PF actually increases 61→62, not decreases; FF drops 26→19. The counts cannot support seven PF→PP transitions. Please correct the transition description and provide a proper transition matrix or a revised interpretation.
  6. [§5.1, Table 1, Table 2] All metrics are point estimates over N=217 pairs with no confidence intervals, bootstrap, or significance tests. BTR numerators are small (e.g., PF≈36 for Claude EA-RAG, PF≈61 for GPT-4o), so a 4–5.5 pp BTR difference may be within sampling noise. For example, approximate 95% Wilson intervals on BTR should be reported. The authors should also report the raw PP/PF/FP/FF counts for Table 1, not only percentages, so that readers can assess precision.
minor comments (6)
  1. [§1, §5.1] The paper describes “four frontier LLMs” but evaluates six models (GLM-5, Kimi K2.5, Nemotron 120B, GPT-4o, GPT-5.4, Claude Sonnet 4.6) from at least five families. Please correct the count.
  2. [Figure 2] The y-axis label “Robustness” is vague; it should say “Acc_rob” and the x-axis “BTR” is already fine. Also clarify whether the standalone markers include GPT-4o RAG or not.
  3. [§4.3] There is a missing period or conjunction in “Self-RAG (Asai et al., 2024) EA-RAG operates entirely at inference time.” Please rephrase.
  4. [References] The reference for Li et al. (2025) appears after Zheng et al. (2023), breaking the alphabetical order. Also consider adding a reference for the embedding model (text-embedding-3-large) and for the GPT-5.4-nano extractor, if these are public artifacts.
  5. [§3.3] The definition of Acc_rob = |PP|/N is fine, but the names “Acc_base” and “Acc_rob” should be introduced with explicit equations. Currently BTR has an equation but the other two do not.
  6. [§4.2] The coverage audit uses cos(ϕ(e_j), ϕ(c_i)) but the embedding dimension is stated for the query side; please clarify that the same encoder is used for both evidence elements and chunks.

Circularity Check

1 steps flagged

No equation-level circularity in the benchmark metrics; the central robustness-gap finding is measured, not derived. The only partial circularity is in-sample fitting of EA-RAG's scaffold and thresholds to the same benchmark used for the reported BTR gains.

specific steps
  1. fitted input called prediction [§4.3 (Stage 3) and §5.2 (Main Results)]
    "The generator receives the narrative, Rfinal, and a scaffold instruction set derived from error taxonomy analysis of 113 baseline failures: ... Missed critical info 48.7% ... EA-RAG achieves 20.3% BTR and 65.0% robust accuracy on Claude Sonnet 4.6, a 5.5 percentage point BTR reduction without degrading base accuracy."

    The scaffold instructions are fitted to the error distribution of the same MamaBench cases on which EA-RAG's BTR reduction is then reported, with θ=0.6 and δ=0.82 hand-set and no held-out split described. The instructions directly dictate the behavior counted by BTR (e.g., 'Analyse ALL clinical parameters' for the dominant 'Missed critical info' failure), so the measured improvement is not an independent prediction; it is partly in-sample. This is partial circularity/overfitting, not a derivation-level identity, and the headline robustness-gap finding does not depend on it.

full rationale

The core claims—Acc_base, Acc_rob, and BTR—are measured conditional frequencies defined by Eq. (2), not quantities derived from fitted parameters; no equation in the paper reduces to its inputs. BTR is adopted from external work (Chen et al., 2026), with no author overlap, so there is no self-citation chain. 'Diagnostic Fixation' is a definition, not a predicted result. The main circularity-adjacent issue is that EA-RAG's scaffold was constructed from an error taxonomy of 113 baseline failures on MamaBench and evaluated on the same benchmark (with hand-set θ, δ and no held-out split), making the reported 5.5 pp BTR gain partly in-sample. The manuscript's own Limitations section concedes the separate LLM-as-judge validity risk, which could inflate BTR, but that is a measurement-validity concern rather than a circular derivation. On balance: central empirical gap is independent and self-contained; the method-evaluation component merits a moderate circularity/overfitting flag, so score 4.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 0 invented entities

The central results rest on several domain assumptions: the judge's reliability, expert ground truth, retrieval-corpus separation, and embedding similarity as clinical evidence. Free parameters θ, δ, and k are hand-set, and the scaffold is designed from failures on the same benchmark. No new physical or clinical entities are introduced.

free parameters (3)
  • coverage threshold θ = 0.6
    EA-RAG Stage 2 uses θ=0.6 to define the gap set; no tuning procedure or dev split is reported.
  • deduplication threshold δ = 0.82
    EA-RAG Stage 2 uses δ=0.82 for deduplication; hand-set without reported validation.
  • retrieval count k and cap k+3 = k=5, cap=8
    EA-RAG uses k=5 top chunks and caps final set at 8; the vanilla RAG baseline uses k=2. No sensitivity analysis is provided.
axioms (5)
  • domain assumption LLM-as-judge (Claude Opus 4.5) correctly classifies open-ended model outputs as clinically correct or wrong.
    Invoked in §3.3; no clinician validation, inter-annotator agreement, or error analysis of the judge is provided.
  • domain assumption Expert-authored base and counterfactual narratives have valid, unambiguous single ground-truth diagnoses.
    §3.2 asserts multi-stage clinical review, but no inter-rater reliability, adjudication details, or release of narratives is available for verification.
  • domain assumption The retrieval corpus contains no MamaBench case narratives or labels.
    §5.1 states this separation, but no audit is provided to confirm zero leakage between benchmark data and retrieval corpus.
  • domain assumption Cosine similarity on text-embedding-3-large embeddings captures clinically meaningful parameter coverage.
    §4.2 uses embeddings for evidence coverage; no clinical validation of similarity thresholds is given.
  • ad hoc to paper The scaffold derived from 113 baseline failures generalizes to all MamaBench pairs.
    §4.3 derives the scaffold from failures on the same benchmark, without a held-out split; improvements may be in-sample.

pith-pipeline@v1.3.0-alltime-deepseek · 7757 in / 16123 out tokens · 153880 ms · 2026-08-02T02:13:30.757366+00:00 · methodology

0 comments
read the original abstract

Large language models achieve strong scores on medical benchmarks, yet these benchmarks evaluate each question in isolation, providing no measure of whether a system can distinguish clinically similar presentations requiring different interventions. We introduce MamaBench, the first counterfactual benchmark for maternal and paediatric AI: 434 expert-authored clinical narratives in 217 pairs across 371 pathologies, evaluated via the Bias Trap Rate (BTR), the conditional probability that a model fails the counterfactual given success on the base case. We propose Evidence-Anchored RAG (EA-RAG), a three-stage retrieval method that replaces aggregate similarity with an evidence coverage objective through clinical parameter extraction, coverage auditing, and contrastive sub-queries. Across eight configurations of four frontier LLMs, base accuracy overstates robust accuracy by 16-28 percentage points in every model. EA-RAG achieves 20.3% BTR and 65.0% robust accuracy on Claude Sonnet 4.6, a 5.5 percentage point BTR reduction without degrading base accuracy. The residual 20% BTR confirms that counterfactual robustness in clinical AI remains an open challenge. Keywords: counterfactual evaluation, clinical AI, maternal healthcare, retrieval-augmented generation, diagnostic robustness

Figures

Figures reproduced from arXiv: 2607.14385 by Abiodun Adereni, Angel Ezendu, Anuoluwa Sotome, Ifeoma Nwabueze, Oluwadarasimi Oguntuase, Oluwafunke Akinbuwa, Oluwasegun Oguntuase, Oluwaseun Odunsi, Samuel Okoko, Thanni Adewuyi.

Figure 1
Figure 1. Figure 1: Standard RAG (left) vs. EA-RAG (right). Standard RAG retrieves top-k chunks by aggregate query similarity, returning near-identical context for base and counterfactual cases (retrieval stagnation). EA-RAG introduces three additional mechanisms: (1) evidence extraction of typed clinical parameters, (2) coverage auditing with contrastive sub-queries to fill gaps, and (3) a taxonomy-grounded generation scaffo… view at source ↗
Figure 2
Figure 2. Figure 2: Bias Trap Rate (BTR) vs. Robustness across different model configurations. The x-axis denotes the Bias Trap Rate (where lower percentages are better), and the y-axis represents Robustness (where higher percentages are better). Standalone models are marked as red circles, standard RAG as blue squares, and the proposed EA-RAG configurations as gold stars. The top-left corner represents the ideal performance … view at source ↗
Figure 3
Figure 3. Figure 3: Pair-level outcome distribution. PF (red) = bias traps: model appears competent but fails under perturbation. EA-RAG shrinks PF while expanding PP [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. IyawoBench v2.0: Extended Diagnostic Evaluation of Large Language Model Clinical Triage in Nigerian Primary Care

    cs.CY 2026-07 reject novelty 4.0

    A new triage benchmark reports that binary safety metrics hide a 77-point under-triage gap for Llama 3.1 8B and that the best LLM changes by deployment scenario, though the paper's formal failure-mode labels are inter...

Reference graph

Works this paper leans on

27 extracted references · 3 linked inside Pith · cited by 1 Pith paper

  1. [1]

    Asai, A., Wu, Z., Wang, Y., Sil, A., & Hajishirzi, H. (2024). Self- RAG : Learning to retrieve, generate, and critique through self-reflection. ICLR

  2. [2]

    Chen, W., Huang, G., Wang, W., & Zhu, Z. (2026). MedEinst : Benchmarking the E instellung Effect in medical LLM s through counterfactual differential diagnosis. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026)

  3. [3]

    Fansi Tchango, A., Goel, R., Wen, Z., Martel, J., & Ghosn, J. (2022). DDXPlus : A new dataset for automatic medical diagnosis. NeurIPS

  4. [4]

    Gardner, M., Artzi, Y., Basmova, V., et al. (2020). Evaluating models' local decision boundaries via contrast sets. Findings of EMNLP, 1307--1323

  5. [5]

    F., Gao, L., et al

    Jiang, Z., Xu, F. F., Gao, L., et al. (2023). Active retrieval augmented generation. EMNLP, 7969--7992

  6. [6]

    Jin, D., Pan, E., Oufattole, N., et al. (2021). What disease does this patient have? Applied Sciences, 11(14), 6421

  7. [7]

    W., & Lu, X

    Jin, Q., Dhingra, B., Liu, Z., Cohen, W. W., & Lu, X. (2019). PubMedQA : A dataset for biomedical research question answering. EMNLP, 2567--2577

  8. [8]

    Karpukhin, V., Oguz, B., Min, S., et al. (2020). Dense passage retrieval for open-domain question answering. EMNLP, 6769--6781

  9. [9]

    Kaushik, D., Hovy, E., & Lipton, Z. C. (2020). Learning the difference that makes a difference with counterfactually-augmented data. ICLR

  10. [10]

    L., et al

    Khattab, O., Santhanam, K., Li, X. L., et al. (2023). Demonstrate- S earch- P redict: Composing retrieval and language models for knowledge-intensive NLP . arXiv:2212.14024

  11. [11]

    Lewis, P., Perez, E., Piktus, A., et al. (2020). Retrieval-augmented generation for knowledge-intensive NLP tasks. NeurIPS, 9459--9474

  12. [12]

    Luchins, A. S. (1942). Mechanization in problem solving: The effect of E instellung. Psych.\ Monographs, 54(6), i--95

  13. [13]

    M., et al

    Nori, H., King, N., McKinney, S. M., et al. (2023). Capabilities of GPT-4 on medical competency examinations. arXiv:2303.13375

  14. [14]

    K., & Sankarasubbu, M

    Pal, A., Umapathi, L. K., & Sankarasubbu, M. (2022). MedMCQA : A large-scale multi-subject multi-choice dataset for medical domain question answering. CHIL, 248--260

  15. [15]

    R., Cole-Lewis, H., Sayres, R., et al

    Pfohl, S. R., Cole-Lewis, H., Sayres, R., et al. (2024). A toolbox for surfacing health equity harms and biases in large language models. arXiv:2403.12025

  16. [16]

    T., Wu, T., Guestrin, C., & Singh, S

    Ribeiro, M. T., Wu, T., Guestrin, C., & Singh, S. (2020). Beyond accuracy: Behavioral testing of NLP models with C heck L ist. ACL, 4902--4912

  17. [17]

    Shi, F., Chen, X., Misra, K., et al. (2023). Large language models can be easily distracted by irrelevant context. ICML

  18. [18]

    Singhal, K., Azizi, S., Tu, T., et al. (2023). Large language models encode clinical knowledge. Nature, 620, 172--180

  19. [19]

    Topol, E. J. (2019). High-performance medicine: The convergence of human and artificial intelligence. Nature Med., 25, 44--56

  20. [20]

    Trivedi, H., Balasubramanian, N., Khot, T., & Sabharwal, A. (2023). Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions. ACL

  21. [21]

    Wang, L., Yang, N., & Wei, F. (2023). Q uery2doc: Query expansion with large language models. EMNLP, 9414--9423

  22. [22]

    Wei, J., Wang, X., Schuurmans, D., et al. (2022). Chain-of-thought prompting elicits reasoning in large language models. NeurIPS, 24824--24837

  23. [23]

    Yao, S., Yu, D., Zhao, J., et al. (2023). Tree of thoughts: Deliberate problem solving with large language models. NeurIPS

  24. [24]

    Zakka, C., Shad, R., Chaurasia, A., et al. (2024). Almanac: Retrieval-augmented language models for clinical medicine. NEJM AI, 1(2)

  25. [25]

    Zheng, L., Chiang, W.-L., Sheng, Y., et al. (2023). Judging LLM -as-a-judge with MT-Bench and C hatbot A rena. NeurIPS

  26. [26]

    Li, M., Zhan, Z., Yang, H., Xiao, Y., Zhou, H., Huang, J., & Zhang, R. (2025). Benchmarking retrieval-augmented large language models in biomedical NLP : Application, robustness, and self-awareness. Science Advances, 11(47), eadr1443

  27. [27]

    Zhan, Z., Wang, J., Zhou, S., Deng, J., & Zhang, R. (2025). MMRAG : multi-mode retrieval-augmented generation with large language models for biomedical in-context learning. Journal of the American Medical Informatics Association, 32(10), 1505--1516