Pith. sign in

REVIEW 4 major objections 5 minor 52 references

Hallucination-Aware Multimodal Benchmark for Gastrointestinal Image Analysis with Large Vision-Language Models

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Finetuning a vision-language model to detect and correct hallucinated sentences produces more accurate gastrointestinal diagnostic reports than finetuning on corrected text alone.

desk verdict A useful new GI hallucination dataset, but the paper's central finetuning claim is confounded by an unspecified, likely asymmetric test-time protocol. read the letter →

arxiv 2505.07001 v2 pith:A46AQLTU submitted 2025-05-11 cs.CV cs.LG

Bidur Khanal , Sandesh Pokhrel , Sanjay Bhandari , Ramesh Rana , Nikesh Shrestha , Ram Bahadur Gurung , Cristian Linte , Angus Watson
show 2 more authors
Yash Raj Shrestha Binod Bhattarai
This is my paper · ORCID
classification cs.CVcs.LG
keywords multimodalmedicaldatasetgastrointestinalimageanalysisvision-languagemodelhallucinationhallucination-awarefinetuningreportgenerationKvasir-v2visualquestionanswering
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Hallucination—model output that does not match the image—plagues vision-language models, and the paper argues that the standard fix, finetuning on corrected reports, leaves accuracy on the table. The paper instead trains models to detect and correct hallucinated sentences, an approach it calls hallucination-aware finetuning. The supporting resource is Gut-VLM, a set of 1,816 gastrointestinal endoscopy images paired with machine-generated draft reports, sentence-level hallucination labels, and expert corrections. Across four open VLMs, hallucination-aware finetuning consistently scores higher than standard report finetuning on the paper's 12-question accuracy metric; for LLaVA-1.6-7B it reaches 90.89 percent versus 83.07 percent. If correct, this offers a cheap route to more reliable medical VLMs: let a general model draft, have experts mark its errors, and train a smaller model to repair them.

What carries the argument

The carrying mechanism is hallucination-aware finetuning: with low-rank adaptation, the model is trained on image-plus-draft inputs where part of the text is marked hallucinated and the target is the expert-corrected report, so the model must first identify the erroneous sentences and then regenerate a corrected version. This contrasts with standard finetuning, where the target is simply the corrected ground-truth report. The dataset supplies the essential supervision: ChatGPT-generated drafts over Kvasir-v2 images, sentence-level hallucination tags, expert corrections, and a diagnostic Q&A format derived from 12 predefined questions, which also powers the QAAS evaluation metric.

What would settle it

Run the released evaluation code and record the exact inputs given to the finetunedH models at test time. If those inputs include a draft report from ChatGPT or the model's own pretrained output, rerun standard finetuning with the same draft input; if the QAAS gap between finetunedH and finetuned disappears, the method's advantage was caused by input asymmetry rather than by hallucination-aware training.

Watch

Extended reading notes

Core claim

Gut-VLM is built by asking a commercial VLM to describe Kvasir-v2 endoscopy images under 12 diagnostic questions, then having expert gastroenterologists mark every hallucinated sentence and provide its correction; only 30.39 percent of the machine-generated reports were fully correct. The paper's central claim is that using these sentence-level tags as training signal—hallucination-aware finetuning—improves test-time report generation more than finetuning on the corrected reports alone. On the Question Answering Accuracy Score (QAAS), which converts each generated response into answers to the same 12 questions and checks them against expert-corrected ground truth, the best hallucination-aware model, LLaVA-1.6-7B, scores 90.89 percent versus 83.07 percent for standard finetuning and 50.89 percent for the pretrained model. The same ordering appears in all four open models tested, and a small expert evaluation also ranks the hallucination-aware variant above standard finetuning.

Load-bearing premise

The comparison assumes both finetuning strategies are evaluated with the same inputs, but the paper never states what the hallucination-aware model receives at test time; if it gets a draft report while the standard model gets only the image, the reported advantage may not be from the training objective.

Editorial extensions

If this is right

  • Across all four open VLMs tested, hallucination-aware finetuning outperforms standard finetuning on QAAS, so the training objective, not a particular model, appears to drive the improvement.
  • Gut-VLM turns hallucination detection into a directly supervised task: sentence-level tags and corrections can be used to train a detector independent of report generation.
  • The two-stage pipeline (machine draft, expert correction) can lower the cost of building medical image-text datasets, since experts review rather than write from scratch.
  • The same annotations power a second task: finetuned on extracted diagnostic Q&A, LLaVA-1.6-7B reaches 87.91 percent QAAS versus 49.26 percent pretrained.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A fair replication should give the hallucination-aware and standard models identical test-time inputs; the paper does not state whether the H-model receives a draft report to correct, so part of the reported gap could come from draft access rather than from the training objective.
  • If correction-learning transfers, the same recipe could be applied to other medical imaging domains where expert text is scarce, such as chest radiography or pathology: generate drafts, have experts tag errors, and finetune to repair them.
  • The sentence-level tags also invite a grounding test: aligning hallucinated sentences with image regions could show whether correction training changes what the model actually attends to, not just what it writes.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces Gut-VLM, a multimodal dataset for gastrointestinal (GI) image analysis built from Kvasir-v2 images. Draft diagnostic reports are generated with ChatGPT-4 Omni and then reviewed and corrected by expert gastroenterologists, yielding sentence-level hallucination tags and corrected reports. The authors propose 'hallucination-aware finetuning,' in which a VLM is finetuned to detect and correct hallucinated sentences rather than to generate reports directly, and they report that this strategy outperforms standard finetuning on four VLMs (LLaVA-1.6-7B, DeepSeek-7B-VL, Qwen2-7B, mPLUG-Owl-2B). Evaluation uses classical metrics (ROUGE-L, BLEU, METEOR), two proposed LLM-based metrics (R-Sim and QAAS), and a small expert evaluation. The headline result is Table 1: LLaVA-1.6-7B finetunedH reaches QAAS 90.89% versus 83.07% for standard finetuning and 50.89% for the pretrained model.

Significance. The Gut-VLM dataset is a potentially useful resource for studying hallucination in medical VLMs, as it is, to my knowledge, the first GI-specific dataset with hallucination tags and expert corrections. The proposed hallucination-aware finetuning idea is interesting and, if validated, could provide a practical alternative to standard finetuning for medical report generation. The benchmark covers four modern VLMs and includes both automatic and expert evaluation, which is a credible effort. However, the paper's central claim rests on a comparison whose test-time protocol is unspecified, and the proposed metrics are computed with the same model family used to create the ground truth, so the results as presented are not yet fully convincing.

major comments (4)
  1. [Section 4 / Section 3.3] The test-time inference protocol for the hallucination-aware (finetunedH) models is never stated. Section 4 says only that the model is finetuned 'to learn to detect hallucinated sentences and then correct them,' but it does not say whether, at generation time, the model receives a draft report (e.g., the pretrained model's own output or ChatGPT's output) to correct, or whether it must produce a corrected report from the image alone. If the H-model receives a draft while the standard finetuned model is prompted with the image only, the 7.8-point QAAS gap in Table 1 is confounded with access to a strong textual prior, and the comparison is not on equal footing. The manuscript must specify the test-time input and, if drafts are used, match the inference protocols or include an ablation in which standard finetuning also receives the same draft.
  2. [Section 5 (QAAS and R-Sim)] The two proposed metrics depend entirely on ChatGPT-4, the same model family that produced the draft reports from which the ground truth was derived. In particular, QAAS first converts each model output into Q&A format using ChatGPT-4; a finetunedH model trained to correct ChatGPT-generated drafts may produce outputs that are lexically and structurally closer to the ChatGPT-generated ground truth, so the ChatGPT-based extractor could favor H-models for reasons unrelated to clinical accuracy. The paper acknowledges single-VLM bias in Section 7, but it does not quantify how much of the reported H-model gain is an artifact of the shared model family. I request an additional analysis that decouples the extractor from the generation family, for example by computing QAAS with a different LLM or by reporting expert evaluation on all models rather than only LLaVA.
  3. [Table 1 / Section 6] All quantitative results are single point estimates with no error bars, confidence intervals, or significance tests. The headline comparison of QAAS 90.89 versus 83.07 could plausibly lie within run-to-run variability of five-epoch LoRA finetuning, especially given the small rank and the modest test set of 366 images. At minimum, the authors should report standard deviations over multiple seeds or a paired statistical test on the test-set predictions, for both the automatic metrics and the expert ratings on the 30-response sample. The paper itself notes the need for significance testing in Section 7, so this is a known gap rather than an acceptable omission.
  4. [Section 3.3] Beyond the test-time protocol, the training data format for hallucination-aware finetuning is underspecified. The paper says the model is finetuned to 'identify hallucinated sentences and correct the response' using the VLM-generated response, tags, and corrected response, but it never shows the concrete input/output template (e.g., how the hallucinated sentences are marked, what the instruction text is, and whether the model is trained to output both tags and corrections). Without this information, the method cannot be reproduced, and the reader cannot tell whether the reported gains come from the correction objective or from a different prompt formatting.
minor comments (5)
  1. [References] References [25] and [26] are duplicates: both cite Liu et al., 'Mitigating hallucination in large multi-modal models via robust instruction tuning.' The duplicate should be removed and the numbering adjusted.
  2. [Section 6] The phrase 'Descriptive F ormat Diagnostic Report' contains an erroneous space in 'F ormat'; this should be corrected.
  3. [Throughout] Model names are written inconsistently: 'LLaVa-1.6-7b' in Table 1 and 'LLaVA-1.6-7B' in the text and elsewhere. Please standardize, and use the correct official casing for each model.
  4. [Figure 1] The statistic 'only 30.39% of the VLM-generated responses are fully correct' (Section 3.2) is reported without defining 'fully correct.' A sentence describing how a response is classified as fully correct versus mixed or fully hallucinated would improve clarity.
  5. [Section 6, Expert Evaluation] The expert evaluation is reported only for LLaVA-1.6-7B and only on 30 responses per model variant; the paper should state explicitly that expert ratings are not available for the other three models and should provide the sampling procedure and any inter-rater or intra-rater measures beyond the coefficient of variation.

Circularity Check

0 steps flagged · score 2.0 of 10

No circularity in the derivation chain; evaluation and protocol ambiguities are validity concerns, not constructional circularity.

full rationale

The paper's central claim—that hallucination-aware finetuning outperforms standard finetuning—is an empirical result measured on a held-out test set, not a quantity derived from its own definitions or fitted parameters. The dataset construction (ChatGPT-generated reports corrected by expert gastroenterologists) and the LLM-assisted metrics (R-Sim, QAAS) introduce potential biases, but these are evaluation-design concerns rather than circularity: the same evaluation protocol is applied to all models, and both the H-model and standard finetuning are trained on the same corrected ground-truth texts. The test-time input for the H-model is underspecified; if it received a draft report while standard finetuning generated from the image alone, the comparison would be confounded, but the paper does not state such a protocol, so no definitional reduction can be exhibited. The paper acknowledges the single-VLM bias in its limitations section. Self-citations appear only in related-work contexts and are not load-bearing for the main claim. No imported uniqueness theorem or ansatz-as-citation is used. Therefore, no step in the claimed derivation chain reduces to its inputs by construction; the identified issues are experimental validity risks, not circularity.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central empirical comparison rests on expert-labeled ground truth and on fixed training hyperparameters. The main assumptions are that expert corrections are accurate, that the 12 MedVQA-GI questions define clinically relevant content, and that one round of ChatGPT draft plus expert correction is a valid basis for training and evaluation. No new physical entities are introduced.

free parameters (5)
  • LoRA rank = 8
    Chosen for all finetuning runs with no sensitivity analysis; the central comparison could depend on this choice.
  • learning rate = 1e-4
    Fixed for all runs; no sweep or sensitivity report is provided.
  • number of epochs = 5
    Fixed until loss stabilized per the paper; no early stopping details or seed variance are reported.
  • test split proportion = 20% per class
    Stratified split chosen by the authors; it affects benchmark difficulty and the numerical results.
  • expert evaluation sample size = 30 responses per model, only LLaVA assessed
    Budget-constrained sample with one inserted consistency sample; it is not a robust error estimate for the reported clinical evaluation.
assumptions (4)
  • domain assumption Expert gastroenterologist corrections are accurate and serve as clinical ground truth.
    The entire dataset and evaluation treat expert corrections as truth. No inter-annotator agreement or independent adjudication is reported in Section 3.2.
  • domain assumption The 12 MedVQA-GI diagnostic questions cover the clinically relevant content of GI endoscopy images.
    Report generation, QAAS, and the Q&A extraction all build on these questions, but the questions are not enumerated in the paper.
  • domain assumption Sentence-level hallucination labels assigned by experts are reliable.
    Labels follow M-HalDetect, but only one expert review round is described, and the discussion acknowledges that sentence-level granularity is a limitation.
  • ad hoc to paper ChatGPT-generated draft reports are a reasonable starting point for expert annotation and finetuning.
    The dataset is defined by corrections of ChatGPT-4 Omni outputs, anchoring all content to one model's style. The paper itself flags this as a limitation in the discussion.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Hallucination-Aware Multimodal Benchmark for Gastrointestinal Image Analysis with Large Vision-Language Models." pith.science (2026). https://pith.science/paper/A46AQLTU

@misc{pith2026250507001,
  author       = {Pith},
  title        = {Pith review of: Hallucination-Aware Multimodal Benchmark for Gastrointestinal Image Analysis with Large Vision-Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/A46AQLTU}},
  note         = {Machine review of arXiv:2505.07001}
}
read the original abstract

Vision-Language Models (VLMs) are becoming increasingly popular in the medical domain, bridging the gap between medical images and clinical language. Existing VLMs demonstrate an impressive ability to comprehend medical images and text queries to generate detailed, descriptive diagnostic medical reports. However, hallucination--the tendency to generate descriptions that are inconsistent with the visual content--remains a significant issue in VLMs, with particularly severe implications in the medical field. To facilitate VLM research on gastrointestinal (GI) image analysis and study hallucination, we curate a multimodal image-text GI dataset: Gut-VLM. This dataset is created using a two-stage pipeline: first, descriptive medical reports of Kvasir-v2 images are generated using ChatGPT, which introduces some hallucinated or incorrect texts. In the second stage, medical experts systematically review these reports, and identify and correct potential inaccuracies to ensure high-quality, clinically reliable annotations. Unlike traditional datasets that contain only descriptive texts, our dataset also features tags identifying hallucinated sentences and their corresponding corrections. A common approach to reducing hallucination in VLM is to finetune the model on a small-scale, problem-specific dataset. However, we take a different strategy using our dataset. Instead of finetuning the VLM solely for generating textual reports, we finetune it to detect and correct hallucinations, an approach we call hallucination-aware finetuning. Our results show that this approach is better than simply finetuning for descriptive report generation. Additionally, we conduct an extensive evaluation of state-of-the-art VLMs across several metrics, establishing a benchmark. GitHub Repo: https://github.com/bhattarailab/Hallucination-Aware-VLM.

Figures

Figures reproduced from arXiv: 2505.07001 by the authors.

Figure 1
Figure 1. Top: Overview of the data annotation pipeline. Bottom: [Left] A sample [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Comparing standard finetuning with hallucination-aware finetuning [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Comparison of VLM responses evaluated across different aspects [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

52 extracted references · 28 canonical work pages

  1. [1]

    arXiv preprint arXiv:2303.08774 (2023)

    Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F.L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al.: GPT-4 technical report. arXiv preprint arXiv:2303.08774 (2023)

    arXiv 2023

  2. [2]

    Gas- troenterology 159(1), 335–349 (2020) 10 Khanal, Pokhrel, Bhandari et al

    Arnold, M., Abnet, C.C., Neale, R.E., Vignat, J., Giovannucci, E.L., McGlynn, K.A., Bray, F.: Global burden of 5 major types of gastrointestinal cancer. Gas- troenterology 159(1), 335–349 (2020) 10 Khanal, Pokhrel, Bhandari et al

    work page 2020

  3. [3]

    Bai, Z., Wang, P., Xiao, T., He, T., Han, Z., Zhang, Z., Shou, M.Z.: Hallucination of multimodal large language models: A survey (2024),https://arxiv.org/abs/ 2404.18930

    arXiv 2024

  4. [4]

    In: Proceedings of the ACL work- shop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization

    Banerjee, S., Lavie, A.: METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In: Proceedings of the ACL work- shop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization. pp. 65–72 (2005)

    work page 2005

  5. [5]

    Sci- entific data 7(1), 283 (2020)

    Borgli, H., Thambawita, V., Smedsrud, P.H., Hicks, S., Jha, D., Eskeland, S.L., Randel, K.R., Pogorelov, K., Lux, M., Nguyen, D.T.D., et al.: Hyperkvasir, a com- prehensive multi-class image and video dataset for gastrointestinal endoscopy. Sci- entific data 7(1), 283 (2020)

    2020

  6. [6]

    Chen, J., Yang, D., Wu, T., Jiang, Y., Hou, X., Li, M., Wang, S., Xiao, D., Li, K., Zhang, L.: Detecting and evaluating medical hallucinations in large vision language models (2024), https://arxiv.org/abs/2406.10185

    arXiv 2024

  7. [7]

    In: Bebis, G., Yin, Z., Kim, E., Bender, J., Subr, K., Kwon, B.C., Zhao, J., Kalkofen, D., Baciu, G

    Chheda, T., Iyer, R., Koppaka, S., Kalbande, D.: Gastrointestinal tract anomaly detection from endoscopic videos using object detection approach. In: Bebis, G., Yin, Z., Kim, E., Bender, J., Subr, K., Kwon, B.C., Zhao, J., Kalkofen, D., Baciu, G. (eds.) Advances in Visual Computing. pp. 494–505. Springer International Pub- lishing, Cham (2020)

    work page 2020

  8. [8]

    DeepSeek-AI, Liu, A., Feng, B., Xue, B., Wang, B., et al.: DeepSeek-V3 technical report (2025), https://arxiv.org/abs/2412.19437

    arXiv 2025

Show all 52 references
  1. [9]

    Gastric Cancer26(2), 275–285 (Mar 2023)

    Du, H., Dong, Z., Wu, L., Li, Y., Liu, J., Luo, C., Zeng, X., Deng, Y., Cheng, D., Diao, W., Zhu, Y., Tao, X., Wang, J., Zhang, C., Yu, H.: A deep-learning based system using multi-modal data for diagnosing gastric neoplasms in real-time (with video). Gastric Cancer26(2), 275–...

    2023

  2. [10]

    Nature630(8017), 625–630 (2024)

    Farquhar, S., Kossen, J., Kuhn, L., Gal, Y.: Detecting hallucinations in large lan- guage models using semantic entropy. Nature630(8017), 625–630 (2024)

    2024

  3. [11]

    In: Proceed- ings of the First International Workshop on Vision-Language Models for Biomedi- cal Applications

    Gautam, S., Storås, A.M., Midoglu, C., Hicks, S.A., Thambawita, V., Halvorsen, P., Riegler, M.A.: Kvasir-VQA: A text-image pair gi tract dataset. In: Proceed- ings of the First International Workshop on Vision-Language Models for Biomedi- cal Applications. p. 3–12. MM ’24, ACM...

    2024

  4. [12]

    In: Proceedings of the AAAI Conference on Artificial Intelligence (2024)

    Gunjal, A., Yin, J., Bas, E.: Detecting and preventing hallucinations in large vision language models. In: Proceedings of the AAAI Conference on Artificial Intelligence (2024)

    2024

  5. [13]

    In: Conference and Labs of the Evaluation Forum (2023)

    Hicks, S., Storås, A.M., Halvorsen, P., de Lange, T., Riegler, M., Thambawita, V.L.: Overview of ImageCLEFmedical 2023 - medical visual question answering for gastrointestinal tract. In: Conference and Labs of the Evaluation Forum (2023)

    2023

  6. [14]

    ICLR1(2), 3 (2022)

    Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W., et al.: LoRA: Low-rank adaptation of large language models. ICLR1(2), 3 (2022)

    2022

  7. [15]

    Hu, H., Zhang, J., Zhao, M., Sun, Z.: CIEM: Contrastive instruction evaluation method for better instruction tuning (2023),https://arxiv.org/abs/2309.02301

    2023 arXiv

  8. [16]

    Jha, D., Ali, S., Emanuelsen, K., Hicks, S.A., VajiraThambawita, Garcia-Ceja, E., Riegler, M.A., de Lange, T., Schmidt, P.T., Johansen, H.D., Johansen, D., Halvorsen, P.: Kvasir-instrument: Diagnostic and therapeutic tool segmentation datasetingastrointestinalendoscopy(2020), ...

    2020 arXiv

  9. [17]

    In: Hallucination-Aware Multimodal Benchmark for GI Analysis 11 Workshop on Machine Learning for Multimodal Healthcare Data

    Jha, D., Sharma, V., Dasu, N., Tomar, N.K., Hicks, S., Bhuyan, M.K., Das, P.K., Riegler, M.A., Halvorsen, P., Bagci, U., et al.: Gastrovision: A multi-class en- doscopy image dataset for computer aided gastrointestinal disease detection. In: Hallucination-Aware Multimodal Benc...

    2023

  10. [18]

    Jiang, Y., Chen, J., Yang, D., Li, M., Wang, S., Wu, T., Li, K., Zhang, L.: CoMT: Chain-of-medical-thought reduces hallucination in medical report gener- ation (2024), https://arxiv.org/abs/2406.11451

    2024 arXiv

  11. [19]

    Jing,L.,Li,R.,Chen,Y.,Du,X.:FaithScore:Fine-grainedevaluationsofhallucina- tions in large vision-language models (2024),https://arxiv.org/abs/2311.01477

    2024 arXiv

  12. [20]

    Diagnostic and interventional imaging (2024)

    Keshavarz, P., Bagherieh, S., Nabipoorashrafi, S.A., Chalian, H., Rahsepar, A.A., Kim, G.H.J., Hassani, C., Raman, S.S., Bedayat, A.: ChatGPT in radiology: A systematic review of performance, pitfalls, and future perspectives. Diagnostic and interventional imaging (2024)

    2024

  13. [21]

    EClinicalMedicine53 (2022)

    Li, J., Hu, S., Shi, C., Dong, Z., Pan, J., Ai, Y., Liu, J., Zhou, W., Deng, Y., Li, Y., et al.: A deep learning and natural language processing-based system for automatic identification and surveillance of high-risk patients undergoing upper endoscopy: a multicenter study. EC...

    2022

  14. [22]

    Li, Y., Du, Y., Zhou, K., Wang, J., Zhao, W.X., Wen, J.R.: Evaluating object hal- lucination in large vision-language models (2023),https://arxiv.org/abs/2305. 10355

    2023

  15. [23]

    In: Text summarization branches out

    Lin, C.Y.: ROUGE: A package for automatic evaluation of summaries. In: Text summarization branches out. pp. 74–81 (2004)

    2004

  16. [24]

    In: Computer vision– ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceedings, part v 13

    Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C.L.: Microsoft coco: Common objects in context. In: Computer vision– ECCV 2014: 13th European conference, zurich, Switzerland, September 6-12, 2014, proceedings, part v 13. pp. 740–755...

    2014

  17. [26]

    org/abs/2306.14565

    Liu, F., Lin, K., Li, L., Wang, J., Yacoob, Y., Wang, L.: Mitigating hallucination in large multi-modal models via robust instruction tuning (2024),https://arxiv. org/abs/2306.14565

    2024 arXiv

  18. [27]

    Liu, H., Li, C., Li, Y., Lee, Y.J.: Improved baselines with visual instruction tuning (2024), https://arxiv.org/abs/2310.03744

    2024 arXiv

  19. [28]

    Advances in neural information processing systems36, 34892–34916 (2023)

    Liu, H., Li, C., Wu, Q., Lee, Y.J.: Visual instruction tuning. Advances in neural information processing systems36, 34892–34916 (2023)

    2023

  20. [29]

    Frontline Gastroenterology 14(4), 306–311 (2023)

    Lorraine-Francis, H., Newberry, E., Aziz, I.: Diagnostic yield of upper gastroin- testinal endoscopy in patients attending a UK centre with symptoms compatible with Rome IV functional dyspepsia. Frontline Gastroenterology 14(4), 306–311 (2023)

    2023

  21. [30]

    GE- Portuguese Journal of Gastroenterology24(6), 269–274 (2017)

    Marques, S., Bispo, M., Pimentel-Nunes, P., Chagas, C., Dinis-Ribeiro, M.: Im- age documentation in gastrointestinal endoscopy: review of recommendations. GE- Portuguese Journal of Gastroenterology24(6), 269–274 (2017)

    2017

  22. [31]

    Cogent Engineering9(1), 2084878 (2022)

    Melaku Bitew Haile, Ayodeji Olalekan Salau, B.E., Belay, A.J.: Detection and classification of gastrointestinal disease using convolutional neural network and svm. Cogent Engineering9(1), 2084878 (2022)

    2022

  23. [32]

    Annual review of psychology68(1), 465–489 (2017)

    Metcalfe, J.: Learning from errors. Annual review of psychology68(1), 465–489 (2017)

    2017

  24. [33]

    In: Proceedings of the 40th annual meeting of the Association for Computational Linguistics

    Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: BLEU: a method for automatic evaluation of machine translation. In: Proceedings of the 40th annual meeting of the Association for Computational Linguistics. pp. 311–318 (2002) 12 Khanal, Pokhrel, Bhandari et al

    2002

  25. [34]

    In: Proceedings of the 8th ACM on Multimedia Systems Conference

    Pogorelov, K., Randel, K.R., Griwodz, C., Eskeland, S.L., de Lange, T., Johansen, D., Spampinato, C., Dang-Nguyen, D.T., Lux, M., Schmidt, P.T., et al.: Kvasir: A multi-class image dataset for computer aided gastrointestinal disease detection. In: Proceedings of the 8th ACM on...

    2017

  26. [35]

    Pokhrel, S., Bhandari, S., Ali, S., Lambrou, T., Nguyen, A., Shrestha, Y.R., Watson, A., Stoyanov, D., Gyawali, P., Bhattarai, B.: NCDD: Nearest centroid distance deficit for out-of-distribution detection in gastrointestinal vision (2024), https://arxiv.org/abs/2412.01590

    2024 arXiv

  27. [36]

    Pokhrel, S., Bhandari, S., Vazquez, E., Lambrou, T., Gyawali, P., Bhattarai, B.: TTA-OOD: Test-time augmentation for improving out-of-distribution detection in gastrointestinal vision (2024),https://arxiv.org/abs/2407.14024

    2024 arXiv

  28. [37]

    Artificial Intelligence in Medicine 143, 102606 (2023)

    Quindós, A., Laiz, P., Vitrià, J., Seguí, S.: Self-supervised out-of-distribution de- tection in wireless capsule endoscopy images. Artificial Intelligence in Medicine 143, 102606 (2023)

    2023

  29. [38]

    Rohrbach, A., Hendricks, L.A., Burns, K., Darrell, T., Saenko, K.: Object halluci- nation in image captioning (2019),https://arxiv.org/abs/1809.02156

    2019 arXiv

  30. [39]

    Scientific Reports 13(1), 4171 (2023)

    Selivanov, A., Rogov, O.Y., Chesakov, D., Shelmanov, A., Fedulova, I., Dylov, D.V.: Medical image captioning via generative pretrained transformers. Scientific Reports 13(1), 4171 (2023)

    2023

  31. [40]

    In: 2022 IEEE 19th India Council Interna- tional Conference (INDICON)

    Sharmila, V., Geetha, S.: Detection and classification of gi-tract anomalies from endoscopic images using deep learning. In: 2022 IEEE 19th India Council Interna- tional Conference (INDICON). pp. 1–6 (Nov 2022)

    2022

  32. [41]

    Scientific Reports14(1), 9330 (2024)

    Shieh, A., Tran, B., He, G., Kumar, M., Freed, J.A., Majety, P.: Assessing Chat- GPT 4.0’s test performance and clinical diagnostic accuracy on USMLE STEP 2 CK and clinical case reports. Scientific Reports14(1), 9330 (2024)

    2024

  33. [42]

    Shrestha, P., Amgain, S., Khanal, B., Linte, C.A., Bhattarai, B.: Medical vision language pretraining: A survey (2023),https://arxiv.org/abs/2312.06224

    2023 arXiv

  34. [43]

    Nature Medicine pp

    Singhal, K., Tu, T., Gottweis, J., Sayres, R., Wulczyn, E., Amin, M., Hou, L., Clark, K., Pfohl, S.R., Cole-Lewis, H., et al.: Toward expert-level medical question answering with large language models. Nature Medicine pp. 1–8 (2025)

    2025

  35. [44]

    Diagnostics13(4) (2023)

    Sivari, E., Bostanci, E., Guzel, M.S., Acici, K., Asuroglu, T., Ercelebi Ayyildiz, T.: A new approach for gastrointestinal tract findings detection and classification: Deep learning-based hybrid stacking ensemble models. Diagnostics13(4) (2023)

    2023

  36. [45]

    Scientific Data8(1), 142 (May 2021)

    Smedsrud, P.H., Thambawita, V., Hicks, S.A., Gjestang, H., Nedrejord, O.O., Næss, E., Borgli, H., Jha, D., Berstad, T.J.D., Eskeland, S.L., Lux, M., Es- peland, H., Petlund, A., Nguyen, D.T.D., Garcia-Ceja, E., Johansen, D., Schmidt, P.T., Toth, E., Hammer, H.L., de Lange, T.,...

    2021

  37. [46]

    Tong, S., Liu, Z., Zhai, Y., Ma, Y., LeCun, Y., Xie, S.: Eyes wide shut? exploring thevisualshortcomingsofmultimodalllms(2024), https://arxiv.org/abs/2401. 06209

    2024

  38. [47]

    Wang, L., He, J., Li, S., Liu, N., Lim, E.P.: Mitigating fine-grained hallucination by fine-tuning large vision-language models with caption rewrites (2023),https: //arxiv.org/abs/2312.01701

    2023 arXiv

  39. [48]

    Wikipedia contributors: Coefficient of variation — Wikipedia, the free encyclope- dia (2025), https://en.wikipedia.org/wiki/Coefficient_of_variation, [On- line; accessed 27-February-2025]

    2025

  40. [49]

    Yang, A., Yang, B., Hui, B., Zheng, B., Yu, B., et al.: Qwen2 technical report (2024), https://arxiv.org/abs/2407.10671 Hallucination-Aware Multimodal Benchmark for GI Analysis 13

    2024 arXiv

  41. [50]

    Ye, J., Xu, H., Liu, H., Hu, A., Yan, M., Qian, Q., Zhang, J., Huang, F., Zhou, J.: mPLUG-Owl3: Towards long image-sequence understanding in multi-modal large language models (2024),https://arxiv.org/abs/2408.04840

    2024 arXiv

  42. [51]

    Yu, Q., Li, J., Wei, L., Pang, L., Ye, W., Qin, B., Tang, S., Tian, Q., Zhuang, Y.: HalluciDoctor: Mitigating hallucinatory toxicity in visual instruction data (2024), https://arxiv.org/abs/2311.13614

    2024 arXiv

  43. [52]

    Yue, Z., Zhang, L., Jin, Q.: Less is more: Mitigating multimodal hallucination from an eos decision perspective (2024),https://arxiv.org/abs/2402.14545

    2024 arXiv

  44. [53]

    Zebra Technologies: Healthcare solutions (2024),https://www.zebra.com/us/en/ industry/healthcare.html, accessed: 2024-02-23

    2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.