Pith. sign in

REVIEW 3 major objections 6 minor 20 references

Health AI models swallow misinformation even when warned

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · glm-5.2

2026-07-09 23:00 UTC pith:2U3XTXDR

load-bearing objection MF degrades accuracy 7.2 pp on average across four lightweight LLMs; LR barely matters. The MF number is likely an upper bound due to TF-IDF claim matching. the 3 major comments →

arxiv 2607.06913 v1 pith:2U3XTXDR submitted 2026-07-08 cs.CY

Evaluating LLM Robustness Under Domain-Specific Prompt Perturbations in Public Health Applications

Chuqing Zhao , Haochen Yang This is my paper
classification cs.CY
keywords healthrobustnesswhenlanguagellmspublicunderapplications
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tests whether four lightweight LLMs maintain accuracy when their inputs are perturbed in ways that reflect real-world public health usage. Two perturbation types are compared: misinformation framing (injecting a false health claim into the prompt) and layperson rewriting (replacing medical terms with everyday language). The central finding is an asymmetry: models are largely resilient to informal vocabulary (1.4 pp average drop) but substantially vulnerable to injected misinformation (7.2 pp average drop, with 9–38% of predictions flipping), even when the false claims are explicitly labeled as unsupported. The paper argues this reveals a deployment risk distinct from the one usually discussed—models do not just fail to understand patients; they get swayed by false beliefs patients carry into their queries.

Core claim

The robustness gap between semantic-level false content injection and lexical-level vocabulary substitution is large and consistent across model families. Lightweight LLMs can bridge professional and lay medical terminology, but they overweight contextual false claims over their own parametric knowledge, even when warned. This means the dominant failure mode in public health AI deployment is not comprehension of informal speech but susceptibility to misinformation carried in by users.

What carries the argument

Two perturbation functions applied to identical test prompts: misinformation framing concatenates a retrieved false health claim (from curated myth-busting datasets) into the prompt, while layperson rewriting applies term-level substitution using a consumer health vocabulary. Accuracy drop and flip rate are measured against a clean baseline across three public health tasks (biomedical QA, clinical reasoning, vaccine stance classification).

Load-bearing premise

The paper assumes that its method of matching misinformation claims to test questions via keyword retrieval produces a realistic simulation of how actual non-clinical users introduce false information into their queries. If the injected claims are more topically salient or directly relevant than what real users would naturally include, the 7.2 pp degradation may overstate the real-world risk.

What would settle it

If a model family showed no accuracy drop under misinformation framing despite dropping under layperson rewriting, the paper's central asymmetry claim would be contradicted. Alternatively, if models that received explicit disclaimers about injected claims showed no residual degradation, the claim that warnings are insufficient would fail.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The manuscript evaluates the robustness of four lightweight LLMs (Llama-3.1-8B, Mistral-7B, Qwen2.5-7B, GPT-4.1-Nano) under two domain-specific prompt perturbations in public health settings: misinformation framing (MF) and layperson rewriting (LR). Using three datasets (PubMedQA, MedQA-USMLE, COVID-19 Vaccine Stance), the authors find that MF causes substantial accuracy degradation (−7.2 pp average) and high flip rates (9–38%), while LR has a comparatively minor effect (−1.4 pp). The paper concludes that these represent distinct deployment risks requiring perturbation-aware evaluation beyond clean benchmarks.

Significance. The study addresses a practically important gap: most LLM evaluations in clinical or public health domains assume expert-authored, well-formed inputs, whereas real-world users introduce misinformation and informal language. The finding that explicit disclaimers do not fully mitigate MF vulnerability is actionable for deployment decisions. The comparison of open-source 7–8B models against a commercial lightweight model provides useful guidance for resource-constrained settings. However, the significance of the quantitative claims is tempered by the small sample size (100 examples per dataset) and the TF-IDF-based claim matching, which may not reflect naturally occurring user misinformation. The benchmark design is reproducible (frozen mapping file, deterministic decoding), which is a strength.

major comments (3)
  1. Section III.B.1: The TF-IDF retrieval method for matching misinformation claims to samples introduces a potential confound. TF-IDF maximizes lexical overlap, meaning injected claims are likely more topically salient than misinformation a real user would naturally introduce. This could inflate the 7.2 pp MF degradation relative to real-world conditions. The paper does not report the semantic relevance distribution of matched claims or compare against a random-matching baseline, making it difficult to distinguish model vulnerability to misinformation from vulnerability to highly-salient contextual injection. A random-matching control or a relevance analysis would substantially strengthen the central claim.
  2. Section IV.A and Table II: The sample size of 100 examples per dataset (3,600 total inference records) is small for drawing robust conclusions, especially when broken down by model, dataset, and condition. For instance, Table II reports per-dataset accuracy changes in single-digit percentage points, which correspond to differences of only 1–4 examples. The confidence intervals in Table I are wide (e.g., Mistral-7B MF: 45.3 [39.8–51.0]), and some per-dataset comparisons in Table II likely lack statistical significance. The paper should either acknowledge this limitation more prominently or expand the evaluation to support the granularity of claims made in the discussion.
  3. Section V.C, Table III, Case 2: The qualitative analysis of hedging as a failure mode is interesting but underdeveloped. The paper states that three models shift from 'yes' to 'maybe' under MF on PubMedQA, but it is unclear how prevalent this pattern is across the full dataset. Without quantifying how often MF induces hedging versus direct answer flips, the qualitative case may not be representative. A frequency count of this failure mode would help assess its importance.
minor comments (6)
  1. Section I: 'lightweighted' should be 'lightweight'.
  2. Section V.A: 'In constrast' should be 'In contrast'.
  3. Section V.A: 'Each dataset represents a distinct tasks' should be 'distinct tasks'.
  4. Section VI: 'MF degradation of -$7.2 pp' contains a stray dollar sign.
  5. Table III: The notation for prediction transitions (e.g., 'B→N×') is compact but could benefit from a clearer legend or expanded explanation in the caption.
  6. Reference [19]: The access date '2025' is vague; a more specific date would improve reproducibility.

Circularity Check

0 steps flagged

No circularity: empirical benchmark with independently defined metrics and externally sourced perturbations

full rationale

This paper is a purely empirical evaluation with no theoretical derivation chain that could be circular. The two core metrics — accuracy drop (∆Acc = Acc(f,D) − Acc(f, δ(D)), Eq. 1) and flip rate (Eq. 2) — are defined independently of any fitted parameters and computed directly from model outputs against ground-truth labels. No parameter is estimated from a subset of data and then 'predicted' on related data. The perturbations (MF claims from COVIDLIES/HealthVer/WHO; LR substitutions from the Consumer Health Vocabulary) are sourced from external datasets, not constructed from the evaluation data itself. The only self-citation is Reference [8] (Liao, Zhao, …, Yang, on cross-attention token pruning), which is topically unrelated to robustness evaluation and is not load-bearing for any claim in this paper. The skeptic's concern about TF-IDF claim-to-sample matching inflating topical relevance is a valid external-validity risk, but it is not circularity: the measured accuracy drop is a genuine empirical observation, not a quantity forced by construction or by a self-referential definition.

Axiom & Free-Parameter Ledger

2 free parameters · 2 axioms · 0 invented entities

The paper introduces no new theoretical entities or parameters fitted to force a result. The free parameters are standard experimental design choices.

free parameters (2)
  • Sample size per dataset = 100
    Chosen to limit compute; affects statistical power.
  • Temperature = 0
    Set for deterministic decoding; standard choice.
axioms (2)
  • domain assumption TF-IDF retrieval accurately matches misinformation claims to relevant samples.
    The paper assumes this matching method (Section III.B.1) creates realistic perturbations without validating the semantic fit.
  • domain assumption Consumer Health Vocabulary covers layperson terminology adequately.
    The LR perturbation (Section III.B.2) relies on this vocabulary being comprehensive for the tested medical terms.

pith-pipeline@v1.1.0-glm · 10717 in / 1636 out tokens · 216362 ms · 2026-07-09T23:00:31.096695+00:00 · methodology

0 comments
read the original abstract

Large language models (LLMs) are increasingly applied in public health applications, yet their robustness to non-clinical user inputs remains underexplored. We propose a domain specific robustness benchmark that evaluates LLMs under two perturbation types that commonly arise when non-clinical users interact with health AI systems: misinformation framing (MF), where prompt might be injected by false health claims, and layperson rewriting (LR), where patients describe symptoms in everyday language rather than medical terminology. Our goal is to evaluate the stability of LLMs under these perturbation. Experiments show that MF degrades accuracy by 7.2 pp on average with prediction flip rates of 9-38 percent, even when claims are explicitly labelled as unsupported; LR causes only 1.4 pp degradation. These findings highlight two distinct deployment risks in public health settings: models may produce incorrect outputs when users unintentionally carry misinformation into their queries, and may misinterpret clinically relevant details when patients use informal language. Both risks call for perturbation-aware robustness evaluation beyond clean baseline benchmark

Figures

Figures reproduced from arXiv: 2607.06913 by Chuqing Zhao, Haochen Yang.

Figure 1
Figure 1. Figure 1: Flip rate (%) under MF and LR perturbations, averaged across three [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

20 extracted references · 20 canonical work pages · 6 internal anchors

  1. [1]

    PubMedQA: A dataset for biomedical research question answering,

    Q. Jin, B. Dhingra, Z. Liu, W. W. Cohen, and X. Lu, “PubMedQA: A dataset for biomedical research question answering,” inProceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, 2019, pp. 2567–2577

    work page 2019

  2. [2]

    Winds of change: Impact of COVID-19 on vaccine-related opinions of twitter users,

    S. Poddar, M. Mondal, J. Misra, N. Ganguly, and S. Ghosh, “Winds of change: Impact of COVID-19 on vaccine-related opinions of twitter users,” inProceedings of the international aaai conference on web and social media, vol. 16, 2022, pp. 782–793

    work page 2022

  3. [3]

    Promptrobust: Towards evaluating the robustness of large language models on adversarial prompts,

    K. Zhu, J. Wang, J. Zhou, Z. Wang, H. Chen, Y . Wang, L. Yang, W. Ye, Y . Zhang, N. Gong, and X. Xie, “Promptrobust: Towards evaluating the robustness of large language models on adversarial prompts,” in Proceedings of the 1st ACM Workshop on Large AI Systems and Models with Privacy and Safety Analysis. ACM, 2024, p. 57–68

    work page 2024

  4. [4]

    RAmBLA: A Framework for Evaluating the Reliability of LLMs as Assistants in the Biomedical Domain

    W. J. Bolton, R. Poyiadzi, E. R. Morrell, G. v. B. G. Bueno, and L. Goetz, “Rambla: A framework for evaluating the reliability of llms as assistants in the biomedical domain,”arXiv preprint arXiv:2403.14578, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  5. [5]

    Making texts in electronic health records comprehen- sible to consumers: A prototype translator,

    Q. Zeng-Treitler, S. Goryachev, H. Kim, A. Keselman, and D. Rosendale, “Making texts in electronic health records comprehen- sible to consumers: A prototype translator,”AMIA Annual Symposium Proceedings, vol. 2007, pp. 836–840, 2007

    work page 2007

  6. [6]

    What disease does this patient have? a large-scale open domain question answering dataset from medical exams,

    D. Jin, E. Pan, N. Oufattole, W.-H. Weng, H. Fang, and P. Szolovits, “What disease does this patient have? a large-scale open domain question answering dataset from medical exams,” inApplied Sciences, vol. 11, no. 14. MDPI, 2021, p. 6421

    work page 2021

  7. [7]

    Large language models in medicine,

    A. J. Thirunavukarasuet al., “Large language models in medicine,” Nature Medicine, vol. 29, no. 8, pp. 1930–1940, 2023. [Online]. Available: https://doi.org/10.1038/s41591-023-02448-8

    work page doi:10.1038/s41591-023-02448-8 1930

  8. [8]

    CATP: Cross-attention token pruning for accuracy preserved multimodal model inference,

    R. Liao, C. Zhao, J. Li, W. Feng, Y . Lyu, B. Chen, and H. Yang, “CATP: Cross-attention token pruning for accuracy preserved multimodal model inference,” in2025 IEEE Conference on Artificial Intelligence (CAI). IEEE, 2025, pp. 1100–1104

    work page 2025

  9. [9]

    Use of large language models as a scalable approach to understanding public health discourse,

    L. Espinosa and M. Salath ´e, “Use of large language models as a scalable approach to understanding public health discourse,”PLOS Digital Health, vol. 3, no. 10, p. e0000631, 2024

    work page 2024

  10. [10]

    Use of large language models to classify epi- demiological characteristics in synthetic and real-world social media posts about conjunctivitis outbreaks: Infodemiology study,

    M. S. Deineret al., “Use of large language models to classify epi- demiological characteristics in synthetic and real-world social media posts about conjunctivitis outbreaks: Infodemiology study,”Journal of Medical Internet Research, vol. 27, p. e65226, 2025

    work page 2025

  11. [11]

    Beyond Accuracy: Measuring Bias Acknowledgment in Chain-of-Thought Reasoning for Responsible AI Evaluation

    X. Sun, W. Gao, Y . Wang, L. Kong, Y . Li, Z. Fan, Z. Zhuang, W. Dong, Z. Zheng, H. Paranjapeet al., “Beyond accuracy: Measuring bias acknowledgment in chain-of-thought reasoning for responsible ai evaluation,”arXiv preprint arXiv:2606.15127, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  12. [12]

    Driftguard: Mitigating asynchronous data drift in federated learning,

    Y . Han, D. Wu, and B. Varghese, “Driftguard: Mitigating asynchronous data drift in federated learning,”arXiv preprint arXiv:2603.18872, 2026

    work page arXiv 2026

  13. [13]

    Interpretable vs Learned Encoders for High-Cardinality Fraud Detection

    X. Han, J. Liu, M. Zheng, Z. Zhang, and C. Wu, “Interpretable vs learned encoders for high-cardinality fraud detection,”arXiv preprint arXiv:2607.00477, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  14. [14]

    COVIDLies: Detecting COVID-19 misinformation on social media,

    T. Hossain, R. L. Logan IV , A. Ugarte, Y . Matsubara, S. Young, and S. Singh, “COVIDLies: Detecting COVID-19 misinformation on social media,” inProceedings of the 1st Workshop on NLP for COVID-19 (Part 2) at EMNLP 2020, K. Verspoor, K. B. Cohen, M. Conway, B. de Bruijn, M. Dredze, R. Mihalcea, and B. Wallace, Eds. Online: Association for Computational Li...

    work page 2020

  15. [15]

    Evidence-based fact-checking of health-related claims,

    M. Sarrouti, A. Ben Abacha, Y . Mrabet, and D. Demner- Fushman, “Evidence-based fact-checking of health-related claims,” inFindings of the Association for Computational Linguistics: EMNLP 2021, M.-F. Moens, X. Huang, L. Specia, and S. W.- t. Yih, Eds. Punta Cana, Dominican Republic: Association for Computational Linguistics, Nov. 2021, pp. 3499–3512. [Onl...

    work page 2021

  16. [16]

    The Llama 3 Herd of Models

    A. Dubeyet al., “The Llama 3 herd of models,”arXiv preprint arXiv:2407.21783, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  17. [17]

    Mistral 7B

    A. Q. Jianget al., “Mistral 7B,”arXiv preprint arXiv:2310.06825, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  18. [18]

    Qwen2.5 Technical Report

    Qwen, A. Yang, B. Yang, B. Zhang, B. Huiet al., “Qwen2.5 technical report,”arXiv preprint arXiv:2412.15115, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  19. [19]

    Introducing GPT-4.1 in the API,

    OpenAI, “Introducing GPT-4.1 in the API,” https://openai.com/index/gpt-4-1/, Apr. 2025, accessed: 2025

    work page 2025

  20. [20]

    SycEval: Evaluating LLM sycophancy,

    A. Fanous, J. Goldberg, A. A. Agarwal, J. Lin, A. Zhou, R. Daneshjou, and S. Koyejo, “SycEval: Evaluating LLM sycophancy,” Feb. 2025, aIES 2025. [Online]. Available: https://arxiv.org/abs/2502.08177

    work page arXiv 2025