Pith. sign in

REVIEW 4 major objections 8 minor 9 cited by

MedHallBench: A New Benchmark for Assessing Hallucination in Medical Large Language Models

T0 review · 4 major / 8 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper argues that medical hallucinations are best measured by counting hallucinated components in captions rather than by text-similarity scores.

desk verdict ACHMI is a relabeled CHAIR metric, the benchmark is unreleased and unspecified, and the evaluation lacks the reliability statistics that would make the metric meaningful; the taxonomy is useful but the paper is not ready for review. read the letter →

arxiv 2412.18947 v4 pith:YI7DUMWT submitted 2024-12-25 cs.CL cs.AI

Kaiwen Zuo , Yirui Jiang This is my paper · ORCID
classification cs.CLcs.AI
keywords medicallargelanguagemodelshallucinationevaluationbenchmarkACHMIchestX-raycaptioningRLHFvision-languageautomaticannotation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that medical hallucinations in vision-language models are better measured by counting hallucinated medical components in generated captions than by traditional text-similarity metrics. It introduces MedHallBench, a benchmark built from expert-validated chest X-ray cases and existing medical question-answer data, plus an automatic annotation pipeline. The central result is ACHMI, an adaptation of the CHAIR metric that computes the fraction of hallucinated medical components (ACHMII) and the fraction of captions containing any hallucination (ACHMIS). Applied to ten models, ACHMI ranks models differently from BLEU, ROUGE, METEOR, and BERTScore, and the paper argues this gives a more nuanced view of clinical hallucination. If right, this gives researchers a scalable way to spot and reduce unsafe model outputs before deployment.

What carries the argument

The load-bearing object is ACHMI (Automatic Caption Hallucination Measurement in Medical Imaging), a medical adaptation of the CHAIR metric. Given an expert-validated list of ground-truth medical findings in a chest X-ray, ACHMI counts every medical component in the model's caption, marks those that do not correspond to ground truth as hallucinations, and forms two ratios: ACHMII (hallucinated components over all generated medical components) and ACHMIS (captions containing at least one hallucinated component over all captions). The metric's work is to separate 'says something wrong' from 'says something off-topic or poorly worded', which traditional n-gram and embedding similarity scores conflate.

What would settle it

Have two independent clinical teams enumerate hallucinated medical components in the same set of model captions using a fixed written taxonomy, then compare their ACHMII and ACHMIS values; low inter-rater agreement or score divergence beyond a pre-set clinical threshold would refute the claim that ACHMI is a stable measurement.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes that hallucination assessment in medical image captioning should be component-level rather than surface-text-level. ACHMI, the proposed metric, asks whether each medical component in a generated caption (an anatomical structure, a pathological finding, a measurement) matches expert-validated findings in the image; it then reports the proportion of hallucinated components and the proportion of captions that contain them. The paper's empirical comparison across ten vision-language models shows that models with respectable lexical-overlap scores can still carry high ACHMI values, and models fine-tuned on the Slake dataset score markedly lower on ACHMI. From this the authors conclude that ACHMI captures hallucination effects that traditional metrics miss and should guide training and fine-tuning of medical LLMs.

Load-bearing premise

Every ACHMI score depends on a written list of 'medical components' that can be counted in a caption and checked against expert ground truth; if two clinical teams cannot reliably produce the same list, the metric's numbers are not stable.

Editorial extensions

If this is right

  • Adopting ACHMI as a reporting standard would let medical imaging benchmarks compare models by their rate of false findings rather than by text quality alone.
  • Fine-tuning and RLHF pipelines can use ACHMI as a reward signal or early-stopping criterion to actively reduce hallucinated findings.
  • Because ACHMI needs only ground-truth component lists, applying the same component-level logic to CT, MRI, or pathology captions is a direct extension of the paper's approach.
  • The published baseline for ten mainstream models gives future work a reference point for measuring hallucination improvement.
  • Using ACHMII and ACHMIS together separates whether a model hallucinates often from whether it hallucinates in every caption, which is information BLEU and ROUGE do not provide.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper names CHAIR as the basis for ACHMI but does not cite it, and it does not publish the component taxonomy, annotation protocol, or inter-rater agreement values; a reader who wants to reuse ACHMI would first need to fix and validate that taxonomy.
  • ACHMI as defined weights every hallucinated component equally, so a missed life-threatening finding and an extra harmless anatomical detail would affect the score the same way; a clinically weighted variant would be safer for deployment decisions.
  • The benchmark mixes international sources such as MIMIC-CXR and MedQA with an stated alignment to mainland China's medical conditions, so users should check whether the component definitions transfer across healthcare systems before generalizing conclusions.
  • A direct testable extension would be to compare ACHMI against expert harm ratings on the same captions: if ACHMI values do not track clinical severity, it should be paired with a severity-weighted measure rather than used alone.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 8 minor

Summary. The paper proposes MedHallBench, a benchmark for evaluating hallucinations in medical vision-language models, together with a new metric, ACHMI (Automatic Caption Hallucination Measurement in Medical Imaging), defined over 'medical components' in generated captions. The authors describe expert and lay evaluation frameworks, an automatic annotation pipeline using active learning and RLHF, and they report an experimental comparison of ten existing models on VQA and image-report-generation tasks using ACHMI, BERTScore, METEOR, ROUGE-1/2/L, and BLEU. The central claim is that ACHMI provides a more nuanced assessment of hallucination than traditional text-matching metrics.

Significance. If the operational details and validation were supplied, this work would address a real gap: hallucination evaluation in medical image captioning. The idea of adapting CHAIR-style object-level hallucination metrics to medical components is promising, and the comparison of ten models is in principle useful. The paper should also be credited for tackling a clinically important problem and for attempting to combine automated scoring with human evaluation. However, the current manuscript does not deliver the central artifact: there is no concrete definition of 'medical components', no annotation protocol with reported reliability, no expert-evaluation results, no dataset statistics or release, and no evidence connecting ACHMI to human judgments. These omissions are load-bearing because ACHMI and the claimed expert validation are the paper's main contributions.

major comments (4)
  1. [Experiment, Evaluation Metrics, Eqs. (1)-(2)] The ACHMI metric is not operationalized. Equations (1) and (2) define ACHMI_I and ACHMI_S as ratios of hallucinated medical components to all medical components and of captions containing hallucinated components to all captions, but the paper does not define what counts as a medical component, how components are extracted from a generated caption, how synonyms and normalization are handled, or how the set of ground-truth components is obtained. The 'Hallucination Annotation' subsection describes a hierarchical classification system and says Cohen's Kappa was measured, but no taxonomy, annotation template, normalization rules, or Kappa values are reported. As a result, the ACHMI values in Table 1 cannot be reproduced or interpreted, and the abstract's claim that ACHMI is more nuanced than BLEU/ROUGE cannot be assessed.
  2. [Methodology, Hallucination Annotation and Framework for Hallucination Evaluation] The expert and lay evaluation components are described programmatically but never reported. The text states that experts use 5-point Likert scales for clinical accuracy, harm severity, and hallucination confidence, that expert and lay evaluations are weighted 0.7 and 0.3, and that inter-rater reliability is measured with Cohen's Kappa, but the paper gives no expert evaluation results, no Kappa values, no annotator counts, no composite scores, and no comparison between expert scoring and automatic ACHMI scoring. The claim that MedHallBench is 'expert-validated' is therefore unsupported by the presented evidence.
  3. [Modeling and Experiment] The RLHF pipeline is disconnected from the experiments. The 'Modeling' section describes a PPO-based RLHF training procedure, but Table 1 evaluates existing models, not a model trained by that pipeline, and no results show that the RLHF-based or active-learning annotation methods produce valid labels. Since these annotation methods are listed among the paper's main contributions, the paper needs either to report results validating the automatic annotation or to remove the claim that the benchmark uses RLHF-based automatic annotation.
  4. [Experiment, Experimental Results Analysis, Table 1] The conclusion that ACHMI is more effective than traditional metrics rests on qualitative inspection of a single table. There are no error bars, no significance tests, no correlation with human judgments, and no case analysis connecting ACHMI to the three announced hallucination types (anatomical, pathological, measurement). To support the claim of a 'more nuanced understanding', the authors should report a comparison of ACHMI with expert ratings and analyze cases where ACHMI and BLEU/ROUGE diverge.
minor comments (8)
  1. [Abstract and throughout] The acronym MLLMs is defined as 'Medical Large Language Models', but MLLM is widely used for 'Multimodal Large Language Models'; please disambiguate the terminology.
  2. [Table 1 and Figure 1] Model names such as 'LLaV A1.5-7b' and 'LLaV A-Med' contain a spurious space; they should be 'LLaVA1.5-7b' and 'LLaVA-Med'.
  3. [Experiment, Models] The sentence 'All the above models were fine-tuned on the Slake (SF) dataset' is confusing, since Table 1 includes models like BLIP2 and MiniGPT4; clarify what 'SF' means for each model and what fine-tuning procedure was used.
  4. [Hallucination Annotation] The text says quality control includes 'statistical monitoring of inter-annotator agreement', but no such statistics appear anywhere in the paper; at least the Cohen's Kappa values should be reported.
  5. [Evaluation Metrics] Equation (2) writes '|{all caption}|'; this should be '|{all captions}|'.
  6. [Introduction, Contributions] The contributions mention an active-learning annotation method, but the Methodology section does not describe it; add a description or remove the claim.
  7. [References] Several references have formatting errors, including 'MIMIC-Ext-MIMIC-CXR-VQA' in the Bae et al. entry; the reference list should be carefully cleaned.
  8. [Conceptualization of MedHallBench] The paper never reports basic dataset statistics, such as the number of images, cases, QA pairs, or generated captions, nor does it state where the dataset will be released; a benchmark paper should provide this information.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: ACHMI is grounded in expert-validated components and the benchmark/RLHF construction is self-contained.

full rationale

The paper's claimed derivation chain is self-contained with respect to circularity. The ACHMI metric defined in Eqs. (1)-(2) counts hallucinated medical components against all medical components, with the components grounded externally: "Given the ground truth objects in the image, ACHMI accurately assesses the performance of captions rather than the images themselves" (Experiment, Evaluation Metrics). The numerator and denominator are therefore external, expert-validated component sets, not outputs of the models being evaluated. The benchmark dataset is assembled from MIMIC-CXR and MedQA plus expert annotation, and the RLHF pipeline uses a standard PPO objective; none of these steps define the evaluation score in terms of the benchmark's own outputs. The claim that ACHMI is more nuanced than traditional metrics is supported by Table 1 rankings, which is a weak form of validation but not a circular reduction. Self-citations to Zuo et al. 2024 and 2025 appear only as general context in the Introduction and are not load-bearing. The main legitimate concern is reproducibility: no medical-component taxonomy, annotation protocol, or reported Cohen's Kappa values are provided, so the ACHMI scores in Table 1 cannot be independently checked. That is a missing-support or correctness risk, not a definitional circularity, under the rule that absent evidence is not equivalent to circular reasoning.

Assumptions & free parameters 1 free parameters · 3 assumptions · 1 invented entities

The central claims rest on the availability and validity of expert annotations, the enumerability of medical components, and the effectiveness of a training pipeline that was never implemented. The weighted scoring parameters are hand-picked.

free parameters (1)
  • Expert/lay evaluation weights = 0.7 and 0.3
    The weighted scoring system in the Hallucination Evaluation section assigns 0.7 to expert and 0.3 to lay evaluations without sensitivity analysis or justification.
assumptions (3)
  • domain assumption Expert annotations provide a valid gold standard for hallucination
    The evaluation framework measures discrepancies against expert descriptions, but no expert qualifications, inter-rater reliability values, or annotation protocol are reported (Methodology section).
  • domain assumption Medical components can be reliably enumerated from captions and images for ACHMI scoring
    ACHMI requires counting hallucinated and total medical components; the paper does not define a component taxonomy, a lexicon, or an annotation procedure (Equations 1-2).
  • ad hoc to paper RLHF with the stated PPO objective reduces hallucinations
    The Modeling section presents LPPO as a training method but no model is trained or tested, so any claim that this pipeline improves annotation or model output is unverified.
invented entities (1)
  • MedHallBench dataset
    purpose: Benchmark for evaluating hallucinations in medical LLMs
    The paper claims a new dataset but provides no release, size, or sample, so there is no external handle to verify it exists.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MedHallBench: A New Benchmark for Assessing Hallucination in Medical Large Language Models." pith.science (2026). https://pith.science/paper/YI7DUMWT

@misc{pith2026241218947,
  author       = {Pith},
  title        = {Pith review of: MedHallBench: A New Benchmark for Assessing Hallucination in Medical Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YI7DUMWT}},
  note         = {Machine review of arXiv:2412.18947}
}
read the original abstract

Medical Large Language Models (MLLMs) have demonstrated potential in healthcare applications, yet their propensity for hallucinations -- generating medically implausible or inaccurate information -- presents substantial risks to patient care. This paper introduces MedHallBench, a comprehensive benchmark framework for evaluating and mitigating hallucinations in MLLMs. Our methodology integrates expert-validated medical case scenarios with established medical databases to create a robust evaluation dataset. The framework employs a sophisticated measurement system that combines automated ACHMI (Automatic Caption Hallucination Measurement in Medical Imaging) scoring with rigorous clinical expert evaluations and utilizes reinforcement learning methods to achieve automatic annotation. Through an optimized reinforcement learning from human feedback (RLHF) training pipeline specifically designed for medical applications, MedHallBench enables thorough evaluation of MLLMs across diverse clinical contexts while maintaining stringent accuracy standards. We conducted comparative experiments involving various models, utilizing the benchmark to establish a baseline for widely adopted large language models (LLMs). Our findings indicate that ACHMI provides a more nuanced understanding of the effects of hallucinations compared to traditional metrics, thereby highlighting its advantages in hallucination assessment. This research establishes a foundational framework for enhancing MLLMs' reliability in healthcare settings and presents actionable strategies for addressing the critical challenge of AI hallucinations in medical applications.

Figures

Figures reproduced from arXiv: 2412.18947 by the authors.

Figure 1
Figure 1. The figure highlights that LLaVA1.5-7b, LLaVA1.5-13b, and MiniGPT-4 are all unable to correctly describe X-ray images, showing a significant deviation from the actual doctor’s response. This illustrates the hallucina￾tion problem of LLMs in the medical field, where the mod￾els fail to provide accurate and reliable interpretations. in medicine(Singhal et al. 2023). The emergence of halluci￾nations within these models… view at source ↗
Figure 2
Figure 2. Methodological Framework for Medical Dataset Construction and Validation. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Iterative Pipeline for Hallucination Detection and Correction in Multimodal Medical Datasets. The process of manual [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: RLHF Training Pipeline for Medical Language [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 9 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. ERRORQUAKE: Heavy-Tailed Error Severity Distributions in Open-Weight Large Language Models

    cs.LG 2026-04 conditional novelty 6.5 of 10

    At matched accuracy, 85 of 210 open-weight LLM pairs have disjoint severity-tail slopes, so error rate alone cannot rank catastrophic-failure risk.

  2. KnowHal: A Knowledge-Driven Benchmark for Comprehensive Multimodal Hallucination Evaluation

    cs.AI 2026-08 conditional novelty 6.0 of 10

    KnowHal is a new benchmark that jointly tests entity, attribute, relation, and knowledge hallucinations in multimodal language models using paired true/false questions on shared images.

  3. MedFactEval and MedAgentBrief: A Framework and Workflow for Generating and Evaluating Factual Clinical Summaries

    cs.CL 2025-09 conditional novelty 6.0 of 10

    An LLM Jury of ten models evaluates whether clinician-defined key facts appear in AI-generated discharge summaries, matching a seven-physician majority as well as a single physician (kappa 0.81 vs 0.67).

  4. TerraMAE: Learning Spatial-Spectral Representations from Hyperspectral Earth Observation Data via Adaptive Masked Autoencoders

    cs.CV 2025-08 reject novelty 5.0 of 10

    The abstract proposes TerraMAE, an adaptive channel-grouping masked autoencoder for hyperspectral Earth observation, but the manuscript body is a different paper, leaving the proposal without any supporting method or ...

  5. A Multi-Task Evaluation of LLMs' Processing of Academic Text Input

    cs.CL 2025-08 unverdicted novelty 4.0 of 10

    The abstract reports Gemini underperforms on four academic text tasks, but the attached full text is an unrelated biomedical retrieval paper, leaving the claims unverifiable.

  6. Trustworthy Medical Imaging with Large Language Models: A Study of Hallucinations Across Modalities

    eess.IV 2025-08 conditional novelty 4.0 of 10

    AI models hallucinate when reading medical images and when generating them from text, producing false findings and anatomically impossible pictures.

  7. KG4Diagnosis: A Hierarchical Multi-Agent LLM Framework with Knowledge Graph Enhancement for Medical Diagnosis

    cs.AI 2024-12 reject novelty 4.0 of 10

    A hierarchical multi-agent LLM framework with automatic knowledge graph construction for diagnosis across 362 diseases is proposed, but no evaluation is reported.

  8. Loki's Dance of Illusions: A Comprehensive Survey of Hallucination in Large Language Models

    cs.CL 2025-06 reject novelty 3.0 of 10

    A survey of LLM hallucination research that formalizes hallucination types and argues, via incompleteness and undecidability arguments, that hallucinations cannot be fully eliminated.

  9. A comprehensive taxonomy of hallucinations in Large Language Models

    cs.CL 2025-08 conditional novelty 2.0 of 10

    A survey that organizes LLM hallucination types, causes, benchmarks, and mitigations, and restates the theorem that hallucination is inevitable for computable LLMs.

Reference graph

Works this paper leans on

36 extracted references · 9 canonical work pages · cited by 9 Pith papers

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Alabi, M.; and Wick, L. 2024. Reinforcement Learning from Human Feedback: Aligning AI Systems with Human Preferences

    work page 2024

  4. [4]

    Bae, S.; Kyung, D.; Ryu, J.; Cho, E.; Lee, G.; Kweon, S.; Oh, J.; JI, L.; Chang, E.; Kim, T.; et al. 2024. MIMIC-Ext-MIMIC-CXR-VQA: A Complex, Diverse, And Large-Scale Visual Question Answering Dataset for Chest X-ray Images

    2024

  5. [5]

    Dai, W.; Li, J.; Li, D.; Tiong, A. M. H.; Zhao, J.; Wang, W.; Li, B.; Fung, P.; and Hoi, S. 2023. InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning. arXiv:2305.06500

    arXiv 2023

  6. [6]

    Fu, C.; Chen, P.; Shen, Y.; Qin, Y.; Zhang, M.; Lin, X.; Qiu, Z.; Lin, W.; Yang, J.; Zheng, X.; Li, K.; Sun, X.; and Ji, R. 2023. MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models. ArXiv, abs/2306.13394

    arXiv 2023

  7. [7]

    Few-shot learning for medical text: A systematic review

    Ge, Y.; Guo, Y.; Yang, Y.-C.; Al-Garadi, M. A.; and Sarker, A. 2022. Few-shot learning for medical text: A systematic review. arXiv preprint arXiv:2204.14081

    work page Pith review arXiv 2022

  8. [8]

    C.; Papaioannou, J.-M.; Grundmann, P.; Oberhauser, T.; L \"o ser, A.; Truhn, D.; and Bressem, K

    Han, T.; Adams, L. C.; Papaioannou, J.-M.; Grundmann, P.; Oberhauser, T.; L \"o ser, A.; Truhn, D.; and Bressem, K. K. 2023. MedAlpaca--an open-source collection of medical conversational AI models and training data. arXiv preprint arXiv:2304.08247

    arXiv 2023

Show all 36 references
  1. [9]

    Huang, Q.; and Zhao, T. 2024. Data Collection and Labeling Techniques for Machine Learning. arXiv preprint arXiv:2407.12793

    2024 arXiv

  2. [10]

    Kim, Y.; Wu, J.; Abdulle, Y.; and Wu, H. 2024. MedExQA: Medical Question Answering Benchmark with Multiple Explanations. arXiv preprint arXiv:2406.06331

    2024 arXiv

  3. [11]

    H.; Cheatham, M.; Medenilla, A.; Sillos, C.; De Leon, L.; Elepa \ n o, C.; Madriaga, M.; Aggabao, R.; Diaz-Candido, G.; Maningo, J.; et al

    Kung, T. H.; Cheatham, M.; Medenilla, A.; Sillos, C.; De Leon, L.; Elepa \ n o, C.; Madriaga, M.; Aggabao, R.; Diaz-Candido, G.; Maningo, J.; et al. 2023. Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models. PLoS digital hea...

    2023

  4. [12]

    Lamurias, A.; Sousa, D.; and Couto, F. M. 2020. Generating biomedical question answering corpora from Q&A forums. IEEE Access, 8: 161042--161051

    2020

  5. [13]

    Li, C.; Wong, C.; Zhang, S.; Usuyama, N.; Liu, H.; Yang, J.; Naumann, T.; Poon, H.; and Gao, J. 2023 a . LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day. arXiv:2306.00890

    2023 arXiv

  6. [14]

    X.; Nie, J.; and rong Wen, J

    Li, J.; Cheng, X.; Zhao, W. X.; Nie, J.; and rong Wen, J. 2023 b . HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models. ArXiv, abs/2305.11747

    2023 arXiv

  7. [15]

    Li, J.; Li, D.; Savarese, S.; and Hoi, S. 2023 c . BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models. arXiv:2301.12597

    2023 arXiv

  8. [16]

    Li, J.; Zhong, S.; and Chen, K. 2021. MLEC-QA: A Chinese multi-choice biomedical question answering dataset. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 8862--8874

    2021

  9. [17]

    Liu, H.; Li, C.; Wu, Q.; and Lee, Y. J. 2023 a . Visual Instruction Tuning. arXiv:2304.08485

    2023 arXiv

  10. [18]

    Liu, Y.; Duan, H.; Zhang, Y.; Li, B.; Zhang, S.; Zhao, W.; Yuan, Y.; Wang, J.; He, C.; Liu, Z.; Chen, K.; and Lin, D. 2023 b . MMBench: Is Your Multi-modal Model an All-around Player? ArXiv, abs/2307.06281

    2023 arXiv

  11. [19]

    K.; and Sankarasubbu, M

    Pal, A.; Umapathi, L. K.; and Sankarasubbu, M. 2023. Med-HALT: Medical Domain Hallucination Test for Large Language Models. arXiv:2307.15343

    2023 arXiv

  12. [20]

    Qian, J.; Jin, Z.; Zhang, Q.; Cai, G.; and Liu, B. 2024. A Liver Cancer Question-Answering System Based on Next-Generation Intelligence and the Large Model Med-PaLM 2. International Journal of Computer Science and Information Technology, 2(1): 28--35

    2024

  13. [21]

    Shi, Y.; Xu, S.; Liu, Z.; Liu, T.; Li, X.; and Liu, N. 2023. Mededit: Model editing for medical question answering with external knowledge bases. arXiv preprint arXiv:2309.16035

    2023 arXiv

  14. [22]

    Singhal, K.; Azizi, S.; Tu, T.; Mahdavi, S.; Wei, J.; Chung, H.; Scales, N.; Tanwani, A.; Cole-Lewis, H.; Pfohl, S.; Payne, P.; Seneviratne, M.; Gamble, P.; Kelly, C.; Babiker, A.; Schärli, N.; Chowdhery, A.; Mansfield, P.; Demner-Fushman, D.; and Natarajan, V. 2023. Large lan...

    2023

  15. [23]

    Sun, Z.; Shen, S.; Cao, S.; Liu, H.; Li, C.; Shen, Y.; Gan, C.; Gui, L.; Wang, Y.-X.; Yang, Y.; Keutzer, K.; and Darrell, T. 2023. Aligning Large Multimodal Models with Factually Augmented RLHF. ArXiv, abs/2309.14525

    2023 arXiv

  16. [24]

    Tan, Z.; Li, D.; Wang, S.; Beigi, A.; Jiang, B.; Bhattacharjee, A.; Karami, M.; Li, J.; Cheng, L.; and Liu, H. 2024. Large language models for data annotation: A survey. arXiv preprint arXiv:2402.13446

    2024 arXiv

  17. [25]

    S.; Cholakkal, H.; Anwer, R

    Thawkar, O.; Shaker, A.; Mullappilly, S. S.; Cholakkal, H.; Anwer, R. M.; Khan, S.; Laaksonen, J.; and Khan, F. S. 2023. Xraygpt: Chest radiographs summarization using medical vision-language models. arXiv preprint arXiv:2306.07971

    2023 arXiv

  18. [26]

    Thirunagalingam, A. 2023. Improving Automated Data Annotation with Self-Supervised Learning: A Pathway to Robust AI Models. International Transactions in Artificial Intelligence, 7(7): 1--22

    2023

  19. [27]

    Tian, Y.; Gan, R.; Song, Y.; Zhang, J.; and Zhang, Y. 2023. ChiMed-GPT: A Chinese Medical Large Language Model with Full Training Regime and Better Alignment to Human Preferences. arXiv preprint arXiv:2311.06025

    2023 arXiv

  20. [28]

    X.; He, H.; Nguyen, T.; Yu, K.-H.; Deng, H.; Brandt, C.; Bitterman, D

    Wang, X.; Zhang, N. X.; He, H.; Nguyen, T.; Yu, K.-H.; Deng, H.; Brandt, C.; Bitterman, D. S.; Pan, L.; Cheng, C.-Y.; et al. 2024. Safety challenges of AI in medicine. arXiv preprint arXiv:2409.18968

    2024 arXiv

  21. [29]

    Wu, C.; Zhang, X.; Zhang, Y.; Wang, Y.; and Xie, W. 2023. Towards Generalist Foundation Model for Radiology by Leveraging Web-scale 2D&3D Medical Data. arXiv:2308.02463

    2023 arXiv

  22. [30]

    J.; and Luo, P

    Xu, P.; Shao, W.; Zhang, K.; Gao, P.; Liu, S.; Lei, M.; Meng, F.; Huang, S.; Qiao, Y. J.; and Luo, P. 2023. LVLM-eHub: A Comprehensive Evaluation Benchmark for Large Vision-Language Models. ArXiv, abs/2306.09265

    2023 arXiv

  23. [31]

    Ye, Q.; Xu, H.; Ye, J.; Yan, M.; Hu, A.; Liu, H.; Qian, Q.; Zhang, J.; Huang, F.; and Zhou, J. 2023. mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaboration. arXiv:2311.04257

    2023 arXiv

  24. [32]

    Zhang, Y.; Ling, H.; Gao, J.; Yin, K.; Lafleche, J.-F.; Barriuso, A.; Torralba, A.; and Fidler, S. 2021. Datasetgan: Efficient labeled data factory with minimal human effort. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 10145--10155

    2021

  25. [33]

    Zhao, W. X.; Zhou, K.; Li, J.; Tang, T.; Wang, X.; Hou, Y.; Min, Y.; Zhang, B.; Zhang, J.; Dong, Z.; Du, Y.; Yang, C.; Chen, Y.; Chen, Z.; Jiang, J.; Ren, R.; Li, Y.; Tang, X.; Liu, Z.; Liu, P.; Nie, J.; and rong Wen, J. 2023. A Survey of Large Language Models. ArXiv, abs/2303.18223

    2023 arXiv

  26. [34]

    Zhu, D.; Chen, J.; Shen, X.; Li, X.; and Elhoseiny, M. 2023. MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models. arXiv:2304.10592

    2023 arXiv

  27. [35]

    Zuo, K.; Jiang, Y.; Mo, F.; and Lio, P. 2024. KG4Diagnosis: A Hierarchical Multi-Agent LLM Framework with Knowledge Graph Enhancement for Medical Diagnosis. arXiv preprint arXiv:2412.16833

    2024 arXiv

  28. [36]

    Zuo, K.; Tang, J.; Qin, H.; Luo, B.; He, L.; and Tang, S. 2025. Satisfactory Medical Consultation based on Terminology-Enhanced Information Retrieval and Emotional In-Context Learning. arXiv:2503.17876

    2025 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.