Trustworthy Medical Imaging with Large Language Models: A Study of Hallucinations Across Modalities

Anindya Bijoy Das; Shahnewaz Karim Sakib; Shibbir Ahmed

REVIEW 4 major objections 4 minor 31 references

Trustworthy Medical Imaging with Large Language Models: A Study of Hallucinations Across Modalities

T0 review · 4 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Current multimodal LLMs hallucinate in both reading medical scans and generating them; a one-line 'for research' justification raises GPT-4o's fabrication of impossible anatomy from 66% to 94%.

desk verdict This paper is a modest empirical survey showing that current multimodal LLMs hallucinate in both medical image interpretation and generation; the qualitative examples convince, but the headline numeric claims, especially the 66%→94% prompt-bypass success rate, rest on under-documented manual evaluation. read the letter →

arxiv 2508.07031 v1 pith:V5BJXFZ5 submitted 2025-08-09 eess.IV cs.AIcs.CV

Anindya Bijoy Das , Shahnewaz Karim Sakib , Shibbir Ahmed This is my paper

classification eess.IVcs.AIcs.CV

keywords hallucinationmultimodalLLMmedicalimagingimage-to-texttext-to-imagechestX-rayCTclassificationpromptsensitivity

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that today's multimodal LLMs cannot yet be trusted for medical imaging: when they read chest X-rays, CTs, or MRIs they both invent findings that are absent and miss findings that are present, and when they generate synthetic scans from text they attach unprompted laterality, surgical artifacts, and anatomically impossible content. The evidence covers two directions—image-to-text interpretation and text-to-image generation—across open-weight and proprietary models. The most direct quantitative result is that rephrasing an implausible prompt with a 'for research purposes' justification raises GPT-4o's success rate at producing clinically impossible images from 66% to 94%. The authors conclude that hallucination-aware evaluation, not raw accuracy, is the appropriate yardstick for clinical safety.

What carries the argument

The central instrument is a paired prompting protocol: each clinically implausible generation request is posed in a direct form (P1) and again with an appended 'for research purposes' justification (P2), which measures how easily model safeguards are bypassed by wording. For interpretation, the load-bearing probe is a set of diagnostic questions on pleural effusion (presence, extent, laterality) applied to chest X-rays, plus zero-shot versus few-shot classification of chest CT scans; these expose the difference between accuracy and hallucination-aware F1 performance.

What would settle it

Have two board-certified radiologists independently score the same 50 implausible-prompt outputs under a pre-registered rubric, and independently relabel the chest X-ray subset for effusion presence, extent, and laterality; if inter-rater agreement is low or the 66%/94% and 30%/46% success rates do not reproduce, the claimed wording-sensitive hallucination rates and the F1-based conclusions collapse.

Watch

Extended reading notes

Core claim

On the interpretation side, the paper reports that LLaVA-v1.5-7B, Gemma-3B, and Qwen2.5-VL-7B hallucinate on pleural-effusion detection in chest X-rays: Qwen shows a high false-negative rate (hallucinated absence) while LLaVA shows a mix of false positives and false negatives, and few-shot prompting in chest-CT cancer classification improves accuracy modestly without eliminating either fabricated or missed findings. On the generation side, GPT-4o and Gemini-2.5 Flash add unprompted clinical details such as right-sided effusions and surgical clips, and across 50 curated clinically implausible prompts they generate anatomically impossible images at rates of 66%/94% (GPT-4o, P1/P2) and 30%/46%

Load-bearing premise

The quantitative claims assume the authors' manual judgments are a correct gold standard: what counts as a hallucinated or omitted finding in the X-ray/CT images, and which of the 50 generated outputs count as successful hallucinations, were decided by the authors alone with no second reader, expert adjudication, or pre-registered rubric.

Editorial extensions

If this is right

Hospitals evaluating LLM triage tools should report false positives and false negatives separately, since models like Qwen can combine moderate accuracy with a high rate of hallucinated absence.
Few-shot examples are not a sufficient remedy: they lift chest-CT classification accuracy a few points but leave the hallucinated-presence/absence profile essentially unchanged.
A generic justification phrase ('for research purposes') can flip a model from refusing an impossible imaging request to complying with it, so safety claims must be tested across prompt paraphrases, not single phrasings.
Generated images carry implicit clinical priors—right-sided effusions, left- or right-sided cysts, surgical clips—that can bias trainees and downstream models if synthetic data are used without explicit labeling.
Accuracy and F1 score disagree substantially in these models, so any deployment threshold should be set on hallucination-aware metrics rather than top-1 accuracy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The P1→P2 jump suggests a general compliance gradient in multimodal LLMs: any plausibility wrapper (research, educational, demonstrative) may relax safety constraints, implying audits should sample a family of paraphrases rather than one prompt.
Because the gold-standard labels for both the curated prompts and the image findings rest on the authors' own judgment, the reported rates are best read as upper bounds on hallucination until an independent radiologist-labeled subset is scored under a pre-registered rubric.
The recurring right-sided laterality in generated effusions hints that training corpora overrepresent that anatomy; a testable extension is to correlate model-chosen side or clip placement with caption statistics in the underlying training data.
The same image-to-text / text-to-image hallucination probe could transfer to other high-stakes visual domains such as pathology slides or fundus photographs, where confident wrong outputs are similarly dangerous and prompt phrasing is easily manipulated.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: the qualitative finding—multimodal LLMs confidently produce clinically impossible content in both interpretation and generation—is real and well illustrated. The toe-fracture chest X-ray and the radioulnar joint-in-abdomen examples speak for themselves. The dual-direction framing (image-to-text plus text-to-image) is convenient but not new; prior benchmarks like MedVH, RadFlag, and MedHallBench already cover this ground. The most novel bit is the prompt-bypass success rate: showing that appending a fake "for research purposes" justification increases generation of impossible anatomy from 66% to 94% for GPT-4o. That phenomenon is worth measuring, and the paper deserves credit for trying.

The problems are with the numbers, not the headline. The internal inconsistency in Sec 5.2—text says Qwen zero-shot CT F1 is 0.06, Table 3 says 0.37—makes you wonder about proofreading. The "Enhanced data filtering" mode is promised in Sec 5.2 but never defined in Sec 4.1.1 and has no column in Table 3. And Table 4's success rates come from 50 manually curated "clinically implausible prompts" judged by the authors alone, with no rubric, no inter-rater reliability, no confidence intervals, and no breakdown of how many prompts were refusals versus generations. With only 50 prompts, moving a few labels changes the percentages by ten points and could erase the claimed P1→P2 gap. The ground-truth derivation for the Indiana chest X-ray and IQ-OTH/NCCD subsets is also not described.

These flaws don't topple the central claim—that LLMs hallucinate in medical imaging is already well established, and the examples alone support it. But they do undermine the specific numeric contributions. The paper reads like a workshop-level empirical note, not a definitive benchmark.

Who is this for? Someone tracking prompt-bypass vulnerabilities in medical image generation might find the P1/P2 comparison a useful datapoint, provided the authors release the prompt list and a codebook. The paper deserves a serious referee because the topic matters and the flaws are fixable, not because it's a breakthrough. I'd send it to peer review with an expectation of major revision: correct the Qwen F1 figure, define the filtering mode, document the curation and labeling protocol, and report raw counts with confidence intervals. If the authors do that, it becomes a legitimate cautionary data point.

Referee Report

4 major / 4 minor

Summary. The paper studies hallucinations in multimodal LLMs applied to medical imaging, covering both image-to-text tasks (pleural effusion detection on chest X-rays, chest CT cancer classification) and text-to-image tasks (generation of clinically implausible medical images). It reports classification accuracy/F1 for three open-source vision-language models and success rates for GPT-4o and Gemini-2.5 Flash in generating implausible content under two prompt styles. The qualitative examples are intended to show common hallucination patterns and prompt-sensitivity.

Significance. If the quantitative results were reproducible, the paper would provide a useful cautionary study of multimodal LLM behavior in medical imaging. The qualitative demonstrations—such as GPT-4o overlaying finger bones on a chest X-ray and both models superimposing unrelated anatomy onto abdominal scans—are compelling and directly support the broad thesis that current models can violate modality-anatomy consistency. A strength is that the study is direct measurement with no fitted parameters, and the qualitative examples give concrete substance to the hallucination categories. However, the quantitative evaluation is under-specified and contains internal inconsistencies, so the specific numeric claims in Tables 3 and 4 are not currently supported as stated.

major comments (4)

[§5.2, Table 3] There is an internal inconsistency: the text states that Qwen's zero-shot F1 score is 0.06 ('Qwen's large gap between accuracy (51.5%) and F1 score (0.06)'), but Table 3 lists Qwen's zero-shot F1 as 0.37. Additionally, the text says classification was conducted in three modes—'Zero-shot, Enhanced data filtering, and Few-shot'—but §4.1.1 defines only zero-shot and few-shot, and Table 3 has no 'Enhanced data filtering' column. Please define this mode, report its results, and correct the F1 inconsistency with the exact sample sizes.
[§5.3, Table 4] The central generative claim—GPT-4o success rates of 66% (P1) and 94% (P2), and Gemini rates of 30%/46%—rests on 50 curated 'clinically implausible prompts' and the authors' own binary judgment of whether each output counts as a successful hallucination. No prompt list, inclusion/exclusion criteria, scoring rubric, inter-rater reliability, or confidence intervals are provided. The paper itself shows that Gemini-2.5 Flash can productively refuse and clarify (Fig. 3), so the distinction between 'hallucinated' and 'refused/corrected' is nontrivial. With n=50, a shift of 5–7 labels changes percentages by 10–14 points and could erase the reported P1→P2 gap. A full evaluation protocol and independent annotation are needed before these numbers can be interpreted.
[§5.1, Table 2] The pleural effusion experiment uses a 'curated subset' of the Indiana Chest X-ray dataset, but the subset size, case selection criteria, and ground-truth label derivation (presence, extent, laterality) are not described. Table 2 reports F1 scores and DQ2 percentages without sample sizes or confidence intervals, and the DQ2 categories are not defined in the caption. Without these details, the quantitative comparisons among LLaVA, Gemma, and Qwen are not reproducible and the claimed severity of each model's hallucination pattern cannot be assessed. Please report the exact number of cases, the label source, and per-class counts.
[§4.2.2, Fig. 7 vs. Table 4] The text says that 'GPT-4o blocks P1 because of its safeguard' (Fig. 7b) yet Table 4 reports a 66% success rate for GPT-4o under P1. This is confusing: if blocking/refusing is counted as failure, the text should say so explicitly; if some P1 attempts still yield images, the example in Fig. 7(b) is not representative. Define the success-rate denominator and clarify whether refusals are scored as failures or as separate outcomes, since this directly affects the Table 4 percentages.

minor comments (4)

[§4.2.1, Figures 4–6] The text says 'Fig. 5 shows that both GPT and Gemini produce images with right-sided effusions,' but the generated chest X-rays appear first in Fig. 4. Please renumber the figures or correct the cross-references so the order matches the narrative.
[Table 2] The column headers 'DQ 2(a) DQ 2(b)' are unclear without a caption explaining the sub-questions. Also, the model names 'LLaVA', 'Gemma', and 'Qwen' are spelled inconsistently (e.g., 'LLaV A' in Table 2, 'Gemma3-4B' in Fig. 8 caption but 'Gemma-3B' in the text).
[§4.2.2] The text alternates between 'GPT-4o' and 'GPT-4' (e.g., 'GPT-4 initially refused'), which could confuse readers. Please use a consistent model identifier throughout.
[General] No code or data availability statement is provided. For a study whose quantitative claims depend on manual curation of prompts and images, releasing the prompt set, output images, and annotation records is essential for reproducibility; at minimum, the paper should state where these can be obtained.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper is an empirical measurement study with no derivation chain, fitted parameters, or imported uniqueness theorems; self-citations are contextual and not load-bearing.

full rationale

This paper is a direct empirical evaluation of hallucination behavior in multimodal LLMs for medical imaging tasks. It contains no mathematical derivation, no fitted parameters, and no first-principles model from which predictions are derived. The central quantitative claims (Tables 2–4) are measurements of model outputs against dataset labels or the authors' manual judgments. No step reduces by construction to its inputs: the pleural-effusion and CT-classification results compare model answers to external dataset ground truth, and the implausible-content 'success rate' simply records whether a model complied with an intentionally impossible prompt, which is an observational quantity rather than a derived one. The few self-citations ([3], [7], [8]) appear only as background context or as support for a side remark about narrative prompts in Sec. 4.2.2; the main hallucination findings do not rest on those citations. The manual curation and labeling in Sec. 5.3 and the internal inconsistency in Sec. 5.2 (Qwen zero-shot F1 reported as 0.06 in text vs. 0.37 in Table 3) are validity/correctness concerns, not circularity: the labels are not constructed from the conclusions they are used to support. Since no load-bearing argument reduces to its own inputs, the circularity score is 0.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper rests on the accuracy of two public datasets, on the authors' subjective labeling of generated outputs, and on unstated inference configurations for the tested models. No free parameters are fitted and no new entities are introduced.

assumptions (3)

domain assumption Dataset labels in Indiana chest X-ray and Iq-Oth/NCCD CT are a correct gold standard for effusion presence/extent/laterality and cancer/normal.
Sec 5.1 and 5.2 use these labels without describing how they were derived or double-checked.
ad hoc to paper The authors' manual judgment of generated images as hallucinated or implausible is a reliable measurement.
Sec 5.3 and Sec 4.2 classify outputs by the authors' own criteria with no inter-rater reliability or expert panel.
domain assumption The queried model versions behave deterministically under the unstated inference settings.
No temperature, decoding parameters, or API versions are reported for the open-source and commercial models.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Trustworthy Medical Imaging with Large Language Models: A Study of Hallucinations Across Modalities." pith.science (2026). https://pith.science/paper/V5BJXFZ5

@misc{pith2026250807031,
  author       = {Pith},
  title        = {Pith review of: Trustworthy Medical Imaging with Large Language Models: A Study of Hallucinations Across Modalities},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/V5BJXFZ5}},
  note         = {Machine review of arXiv:2508.07031}
}

read the original abstract

Large Language Models (LLMs) are increasingly applied to medical imaging tasks, including image interpretation and synthetic image generation. However, these models often produce hallucinations, which are confident but incorrect outputs that can mislead clinical decisions. This study examines hallucinations in two directions: image to text, where LLMs generate reports from X-ray, CT, or MRI scans, and text to image, where models create medical images from clinical prompts. We analyze errors such as factual inconsistencies and anatomical inaccuracies, evaluating outputs using expert informed criteria across imaging modalities. Our findings reveal common patterns of hallucination in both interpretive and generative tasks, with implications for clinical reliability. We also discuss factors contributing to these failures, including model architecture and training data. By systematically studying both image understanding and generation, this work provides insights into improving the safety and trustworthiness of LLM driven medical imaging systems.

Figures

Figures reproduced from arXiv: 2508.07031 by the authors.

**Figure 2.** Chest X-ray (a) PA view and (b) lateral view with bilateral [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗

**Figure 3.** Model responses to the prompt: “Generate a chest X [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗

Figures from the paper (4 more)

**Figure 4.** Figure 4: Chest X-rays generated by (a) GPT-4o and (b) Gemini-2.5 [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]

**Figure 5.** Figure 5: LLMs generating abdominal CT for renal cyst: (a) GPT [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]

**Figure 7.** Figure 7: Model responses to hallucination prompts: (a) Gemini-2.5 [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]

**Figure 8.** Figure 8: Confusion matrices for DQ 1 to detect pleural effusions with (a) LLaVA-v1.5-7B (b) Gemma3-4B, and (c) Qwen2.5-7B [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

31 extracted references · 21 canonical work pages

[1]

GPT-4 technical report

Josh Achiam, Steven Adler, Sandhini Agarwal, et al. GPT-4 technical report. preprint, arXiv:2303.08774, 2023. 2

arXiv 2023
[2]

Creating trustworthy LLMs: Dealing with hallucinations in healthcare AI

Muhammad Aurangzeb Ahmad, Ilker Yaramis, and Taposh Dutta Roy. Creating trustworthy LLMs: Dealing with hallucinations in healthcare AI. preprint, arXiv:2311.01463,

arXiv
[3]

Can large language models challenge CNNs in medical image analysis? In IEEE International Conference on Image Processing (ICIP), 2025

Shibbir Ahmed, Shahnewaz Karim Sakib, and Anindya Bijoy Das. Can large language models challenge CNNs in medical image analysis? In IEEE International Conference on Image Processing (ICIP), 2025. 1

work page 2025
[4]

Al-Yasriy

Hamdalla F. Al-Yasriy. The iq-oth/nccd lung cancer dataset. Kaggle, 2020. https://www.kaggle.com/datasets/ hamdallak/the-iqothnccd-lung-cancer-dataset . 7

work page 2020
[5]

A framework to assess clini- cal safety and hallucination rates of LLMs for medical text summarisation

Elham Asgari, Nina Montaña-Brown, Magda Dubois, Saleh Khalil, Jasmine Balloch, et al. A framework to assess clini- cal safety and hallucination rates of LLMs for medical text summarisation. npj Digital Medicine, 8(1):1–15, 2025. 1, 3

work page 2025
[6]

Reducing hallucinations of medical multi- modal large language models with visual retrieval-augmented generation

Yun-Wei Chu, Kai Zhang, Christopher Malon, and Mar- tin Renqiang Min. Reducing hallucinations of medical multi- modal large language models with visual retrieval-augmented generation. preprint, arXiv:2502.15040, 2025. 3

arXiv 2025
[7]

Breaking the shield: Vulnerabilities in content moderation for multi- modal language models

Anindya Bijoy Das and Shahnewaz Karim Sakib. Breaking the shield: Vulnerabilities in content moderation for multi- modal language models. Authorea Preprints, 2025. 3, 6

work page 2025
[8]

Hallucinations and Key Information Extraction in Medical Texts: A Comprehensive Assessment of Open-Source Large Language Models

Anindya Bijoy Das, Shibbir Ahmed, and Shahnewaz Karim Sakib. Hallucinations and key information extraction in med- ical texts: A comprehensive assessment of open-source large language models. preprint, arXiv:2504.19061, 2025. 1

work page Pith review arXiv 2025

Show all 31 references

[9]

Potential of ChatGPT and GPT-4 for data mining of free-text CT reports on lung cancer

Matthias A Fink, Arved Bischoff, et al. Potential of ChatGPT and GPT-4 for data mining of free-text CT reports on lung cancer. Radiology, 308(3):e231362, 2023. 1, 4

2023
[10]

Medvh: Towards systematic evaluation of hallucination for large vision language models in the medical context

Zishan Gu, Changchang Yin, Fenglin Liu, and Ping Zhang. Medvh: Towards systematic evaluation of hallucination for large vision language models in the medical context. preprint, arXiv:2407.02730, 2024. 3

2024 arXiv
[11]

Rextrust: A model for fine-grained hallucina- tion detection in AI-generated radiology reports

Romain Hardy, Sung Eun Kim, Du Hyun Ro, and Pranav Rajpurkar. Rextrust: A model for fine-grained hallucina- tion detection in AI-generated radiology reports. preprint, arXiv:2412.15264, 2024. 1, 3

2024 arXiv
[12]

FactCheXcker: Miti- gating measurement hallucinations in chest X-ray report gen- eration models

Alice Heiman, Xiaoman Zhang, et al. FactCheXcker: Miti- gating measurement hallucinations in chest X-ray report gen- eration models. In Proceedings of the Computer Vision and Pattern Recognition Conf., pages 30787–30796, 2025. 3

2025
[13]

DALL-M: Context-aware clinical data augmentation with large language models

Chihcheng Hsieh, Catarina Moreira, Isabel Blanco Nobre, Sandra Costa Sousa, et al. DALL-M: Context-aware clinical data augmentation with large language models. Computers in Biology and Medicine, 190:110022, 2025. 3

2025
[14]

Evaluation of SVM performance in the detection of lung cancer in marked ct scan dataset

Hamdalla F Kareem, Muayed S AL-Husieny, Furat Y Mohsen, Enam A Khalil, and Zainab S Hassan. Evaluation of SVM performance in the detection of lung cancer in marked ct scan dataset. Indonesian Journal of Electrical Engineering and Computer Science, 21(3):1731, 2021. 7

2021
[15]

Tackling hallucination from conditional models for medical image reconstruction with dynamicdps

Seunghoi Kim, Henry FJ Tregidgo, Matteo Figini, Chen Jin, Sarang Joshi, and Daniel C Alexander. Tackling hallucination from conditional models for medical image reconstruction with dynamicdps. preprint, arXiv:2503.01075, 2025. 3

2025 arXiv
[16]

Medical hallucinations in foundation models and their impact on healthcare

Yubin Kim, Hyewon Jeong, Shan Chen, Shuyue Stella Li, et al. Medical hallucinations in foundation models and their impact on healthcare. preprint, arXiv:2503.05777, 2025. 3

2025
[17]

Mitigating structural hallucination in LLMs with local diffusion

Kizuki Kiritani and Tsumugi Kayano. Mitigating structural hallucination in LLMs with local diffusion. 2024. 3

2024
[18]

LLM-CXR: instruction- finetuned LLM for CXR image understanding and generation

Suhyeon Lee, Won Jun Kim, et al. LLM-CXR: instruction- finetuned LLM for CXR image understanding and generation. preprint, arXiv:2305.11490, 2023. 7

2023 arXiv
[19]

Towards a holistic framework for multimodal LLM in 3D brain CT radiology report generation

Cheng-Yi Li, Kao-Jung Chang, Cheng-Fu Yang, Hsin-Yu Wu, Wenting Chen, et al. Towards a holistic framework for multimodal LLM in 3D brain CT radiology report generation. Nature Communications, 16(1):2258, 2025. 1

2025
[20]

Prompt-guided generation of structured chest X-ray report using a pre-trained LLM

Hongzhao Li, Hongyu Wang, Xia Sun, Hua He, and Jun Feng. Prompt-guided generation of structured chest X-ray report using a pre-trained LLM. In IEEE International Conference on Multimedia and Expo (ICME), pages 1–6, 2024. 2, 6

2024
[21]

Addressing image hal- lucination in text-to-image generation through factual image retrieval

Youngsun Lim and Hyunjung Shim. Addressing image hal- lucination in text-to-image generation through factual image retrieval. preprint, arXiv:2407.10683, 2024. 3

2024 arXiv
[22]

Indiana university chest x- ray

National Library of Medicine. Indiana university chest x- ray. Kaggle, 2020. https://www.kaggle.com/datasets/ raddar/chest-xrays-indiana-university . 6

2020
[23]

Med-HALT: Medical domain hallucination test for LLMs

Ankit Pal, Logesh Kumar Umapathi, and Malaikannan Sankarasubbu. Med-HALT: Medical domain hallucination test for LLMs. preprint, arXiv:2307.15343, 2023. 1

2023 arXiv
[24]

Leveraging large language models to foster equity in health- care

Jorge A Rodriguez, Emily Alsentzer, and David W Bates. Leveraging large language models to foster equity in health- care. Journal of the American Medical Informatics Associa- tion, 31(9):2147–2150, 2024. 3

2024
[25]

Gemini: a family of highly capable multimodal models

Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean- Baptiste Alayrac, et al. Gemini: a family of highly capable multimodal models. preprint, arXiv:2312.11805, 2023. 2

2023 arXiv
[26]

Hallucination index: An image quality metric for gener- ative reconstruction models

Matthew Tivnan, Siyeop Yoon, Zhennong Chen, Xiang Li, et al. Hallucination index: An image quality metric for gener- ative reconstruction models. In International Conference on Medical Image Computing and Computer-Assisted Interven- tion, pages 449–458. Springer, 2024. 3

2024
[27]

On large visual language models for medical imaging analysis: An empirical study

Minh-Hao Van, Prateek Verma, and Xintao Wu. On large visual language models for medical imaging analysis: An empirical study. In IEEE/ACM Conference on Connected Health: Applications, Systems and Engineering Technologies (CHASE), pages 172–176, 2024. 4

2024
[28]

Rodney Long, and George R

Zhiyun Xue, Daekeun You, Sema Candemir, Stefan Jaeger, Sameer Antani, L. Rodney Long, and George R. Thoma. Chest X-ray image view classification. In IEEE 28th Inter- national Symposium on Computer-Based Medical Systems , pages 66–71, 2015. 6

2015
[29]

Med-hvl: Au- tomatic medical domain hallucination evaluation for large vision-language models

Qianqi Yan, Xuehai He, and Xin Eric Wang. Med-hvl: Au- tomatic medical domain hallucination evaluation for large vision-language models. In AAAI 2024 Spring Symposium on Clinical Foundation Models, 2024. 2, 3

2024
[30]

Radflag: A black-box hallucination detection method for medical vision language models

Serena Zhang, Sraavya Sambara, Oishi Banerjee, Julian Acosta, L John Fahrner, and Pranav Rajpurkar. Radflag: A black-box hallucination detection method for medical vision language models. preprint, arXiv:2411.00299, 2024. 3

2024 arXiv
[31]

MedHallBench: A new bench- mark for assessing hallucination in medical large language models

Kaiwen Zuo and Yirui Jiang. MedHallBench: A new bench- mark for assessing hallucination in medical large language models. preprint, arXiv:2412.18947, 2024. 3

2024 arXiv

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.

[1] [1]

GPT-4 technical report

Josh Achiam, Steven Adler, Sandhini Agarwal, et al. GPT-4 technical report. preprint, arXiv:2303.08774, 2023. 2

arXiv 2023

[2] [2]

Creating trustworthy LLMs: Dealing with hallucinations in healthcare AI

Muhammad Aurangzeb Ahmad, Ilker Yaramis, and Taposh Dutta Roy. Creating trustworthy LLMs: Dealing with hallucinations in healthcare AI. preprint, arXiv:2311.01463,

arXiv

[3] [3]

Can large language models challenge CNNs in medical image analysis? In IEEE International Conference on Image Processing (ICIP), 2025

Shibbir Ahmed, Shahnewaz Karim Sakib, and Anindya Bijoy Das. Can large language models challenge CNNs in medical image analysis? In IEEE International Conference on Image Processing (ICIP), 2025. 1

work page 2025

[4] [4]

Al-Yasriy

Hamdalla F. Al-Yasriy. The iq-oth/nccd lung cancer dataset. Kaggle, 2020. https://www.kaggle.com/datasets/ hamdallak/the-iqothnccd-lung-cancer-dataset . 7

work page 2020

[5] [5]

A framework to assess clini- cal safety and hallucination rates of LLMs for medical text summarisation

Elham Asgari, Nina Montaña-Brown, Magda Dubois, Saleh Khalil, Jasmine Balloch, et al. A framework to assess clini- cal safety and hallucination rates of LLMs for medical text summarisation. npj Digital Medicine, 8(1):1–15, 2025. 1, 3

work page 2025

[6] [6]

Reducing hallucinations of medical multi- modal large language models with visual retrieval-augmented generation

Yun-Wei Chu, Kai Zhang, Christopher Malon, and Mar- tin Renqiang Min. Reducing hallucinations of medical multi- modal large language models with visual retrieval-augmented generation. preprint, arXiv:2502.15040, 2025. 3

arXiv 2025

[7] [7]

Breaking the shield: Vulnerabilities in content moderation for multi- modal language models

Anindya Bijoy Das and Shahnewaz Karim Sakib. Breaking the shield: Vulnerabilities in content moderation for multi- modal language models. Authorea Preprints, 2025. 3, 6

work page 2025

[8] [8]

Hallucinations and Key Information Extraction in Medical Texts: A Comprehensive Assessment of Open-Source Large Language Models

Anindya Bijoy Das, Shibbir Ahmed, and Shahnewaz Karim Sakib. Hallucinations and key information extraction in med- ical texts: A comprehensive assessment of open-source large language models. preprint, arXiv:2504.19061, 2025. 1

work page Pith review arXiv 2025