Pith. sign in

REVIEW 5 major objections 4 minor 35 references

MIRA: A Novel Framework for Fusing Modalities in Medical RAG

T0 review · 5 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read MIRA is a retrieval-augmented generation framework that grounds medical answers in retrieved images and text, using a rethink-rearrange loop to cut hallucination and factual error.

desk verdict A real engineering pipeline whose headline claims are contradicted by its own tables; worth engaging on revision, not as it stands. read the letter →

arxiv 2507.07902 v1 pith:GLCS44WQ submitted 2025-07-10 cs.CV

Jinhong Wang , Tajamul Ashraf , Zongyan Han , Jorma Laaksonen , Rao Mohammad Anwer This is my paper
classification cs.CV
keywords retrieval-augmentedgenerationmedicalvisualquestionansweringmultimodallargelanguagemodelshallucinationmitigationchain-of-thoughtreasoningradiologyreportfactualaccuracydynamicretrieval
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces MIRA, a retrieval-augmented generation framework for medical visual question answering built to reduce factual errors and hallucination. Its central claim is that an 8-billion-parameter vision-language model can reach clinically useful accuracy by retrieving both text and images from offline medical databases and live web search, dynamically choosing how many retrieved contexts to include, and running a rethink-rearrange loop in which the model writes an initial answer, critiques it against the retrieved evidence, and rewrites a final answer. The paper reports that this design outperforms earlier medical vision-language models on a 1,000-image MIMIC-CXR report generation subset and approaches the accuracy of 72-billion-parameter frontier models on PMC-VQA while claiming roughly nine times faster inference. The motivation matters because medical question answering can surface fabricated findings or unsupported treatment advice when a model relies only on its internal knowledge.

What carries the argument

The load-bearing mechanism is the Rethink-Rearrange (RTRA) module: a three-stage validation loop in which the model first drafts an answer from retrieved multimodal evidence, then critiques that answer via chain-of-thought reasoning against the evidence, and finally writes a refined response, with a filtering step that drops retrieved chunks judged irrelevant while keeping at least one text and one image reference. Around it sits a multimodal retrieval pipeline that encodes queries and database entries in a shared embedding space, retrieves from both an offline medical database and an online search API, and fuses the two modalities through a learned attention weight so that image-centric and text-centric questions are handled differently. A query rewrite module rewrites user questions before encoding, and a reinforcement-style reward combining a factual score and a coherence score steers the refinement.

What would settle it

Check for near-duplicate questions or images between the 50k PubMedVision/VQA-Med fine-tuning instances described in Section 3.3 and the PMC-VQA test set, then rerun inference on the non-overlapping subset; if the 0.66 and 0.39 scores drop substantially, the reported generalization is overstated. For the report-generation claim, rerun on a random 1,000-image MIMIC-CXR sample and compare against the available MMed-RAG system, which the paper notes was excluded because its code and weights were unavailable.

Watch

Extended reading notes

Core claim

MIRA is a multimodal RAG framework whose central claim is that medical answers should be grounded in retrieved multimodal evidence rather than memory alone, with the amount of evidence adapted per query. The multimodal retrieval path jointly pulls three text chunks, two images, and one online text paragraph into a joint embedding space, then a dynamic attention mechanism weights image and text contributions according to the query. The Rethink-Rearrange module then generates an initial answer, runs a chain-of-thought self-critique against the retrieved material, filters out irrelevant retrieved chunks, and produces a final refined answer. The paper reports that with no pretraining on PMC-VQA, this recipe scores 0.66 on true/false questions and 0.39 on detailed questions on PMC-VQA, outperforming earlier medical vision-language models while trailing two much larger systems, and that it achieves the best scores among the compared models on the MIMIC-CXR report generation subset. The authors describe the results as substantially improving factual accuracy and establishing new state-of-the-art performance.

Load-bearing premise

The evaluation-generalization claim depends on the assumption that the PMC-VQA test set is not contaminated by the fine-tuning data drawn from PubMedVision, so the reported accuracy measures out-of-distribution ability rather than memorization.

Editorial extensions

If this is right

  • If MIRA works as described, medical VQA systems can reduce hallucination risk by conditioning every answer on retrieved evidence and by deleting irrelevant retrieved chunks before generation.
  • The online search path means the knowledge base is not frozen, so recommendations can reflect current guidelines, though the quality of retrieved web sources then becomes part of the system's reliability.
  • The 8-billion-parameter scale with roughly nine times faster inference than 72-billion-parameter models suggests evidence-grounded medical assistants could run at interactive speeds without a frontier-model budget.
  • The paper's preference analysis, in which MIRA is preferred by an external judge model in 46% of comparisons against one large model and 59.5% against another, indicates the design is competitive in open-ended clinical questions, not just closed-form VQA.
  • The reported gains on MIMIC-CXR report generation imply that the same retrieval-and-rethink loop can be applied to radiology reporting, not only to short-answer questions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The state-of-the-art claim rests mainly on the MIMIC-CXR subset, since the paper's own PMC-VQA table shows MIRA trailing two much larger models, and the paper notes that the most relevant RAG baseline, MMed-RAG, was excluded because its code and weights were unavailable.
  • The statement that MIRA had no pretraining on PMC-VQA is weaker than a no-overlap guarantee: the fine-tuning set is curated from PubMedVision, and PMC-VQA is built from the same source, so a leakage check on near-duplicate questions or images would settle whether the reported accuracy measures generalization.
  • The appendix shows online-RAG snippets containing clearly irrelevant web-page text, such as navigation artifacts, suggesting the online retrieval path can inject noise into the evidence; measuring retrieval precision against end-task accuracy would be a natural extension.
  • The dynamic-k claim could be made causal by comparing fixed-k retrieval settings against the Rearrange-filtered adaptive k on the same evaluation splits, since the paper's ablations vary the online/offline path and the modality but not the number of retrieved chunks directly.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper proposes MIRA, a multimodal retrieval-augmented generation framework for medical VQA and report generation. MIRA combines a LLaVA-style vision-language backbone with offline and online retrieval, a query-rewrite module, and a 'Rethink-Rearrange' (RTRA) module that iteratively selects and refines retrieved evidence through chain-of-thought reasoning. The manuscript claims new state-of-the-art results on medical benchmarks, a 9x inference speedup over 72B models, and the first multimodal RAG framework for MedVQA. Evaluations are reported on MIMIC-CXR report generation and PMC-VQA, with ablations on retrieval components and a preference study.

Significance. If the claims were supported, MIRA would be a practically useful contribution: dynamic retrieval selection and iterative refinement are reasonable mechanisms for grounding medical generation, and the paper releases code, which aids reproducibility. The paper also provides ablations (Table 5) showing the value of both online and offline retrieval pathways. However, the central performance claim is contradicted by the paper's own results, and the evaluation design contains a training/evaluation source-overlap risk. The contribution is therefore not currently established at the level required for the claimed state-of-the-art status.

major comments (5)
  1. [Abstract / Table 3] The abstract and Section 1 claim 'new state-of-the-art results,' but Table 3 shows MIRA trailing both Qwen2.5-VL-72B and OpenAI o1 on both PMC-VQA splits: 0.66 vs 0.69 conversational and 0.39 vs 0.47/0.49 detailed. Section 5.5 acknowledges this gap, so the unqualified SOTA claim is unsupported by the manuscript's own evidence. The claim should either be removed or restricted to same-scale models.
  2. [Section 5.4 / Table 2] The MIMIC-CXR comparison is run on a 1,000-sample subset and omits MMed-RAG and RULE, the two most relevant RAG baselines cited in Section 2. The included baselines are non-RAG or zero-shot models, so Table 2 cannot support a state-of-the-art claim among medical RAG systems. The statement in Section 5.4 that MMed-RAG was excluded due to unavailable code does not justify claiming superiority; at minimum, results from published papers should be reported for these baselines or the SOTA claim should be withdrawn.
  3. [Section 3.3 / Section 5.5] The fine-tuning set described in Section 3.3 includes 50k instances curated from PubMedVision and VQA-Med, while the main evaluation in Section 5.5 uses PMC-VQA, which is built from the same source. The statement that MIRA had 'no pretraining on PMC-VQA' does not rule out overlap or near-duplicates between the training data and the test set. The paper needs an explicit overlap analysis or an evaluation on a disjoint held-out set; without this, the reported PMC-VQA numbers may reflect memorization rather than generalization.
  4. [Section 3.3 / Section 3.4] Equations (7) and (8) define a reward function and a policy-gradient update for 'Reinforced CoT Generation,' but Section 3.4 states that 'All training stages are using cross-entropy loss' and gives Eq. (9) as the only training objective. This is an internal contradiction. The authors must clarify whether RL is actually used; if so, the RL objective and training details must be reported, and if not, Eqs. (7)-(8) should be removed.
  5. [Section 1 / Section 5.5] The paper repeatedly claims a '9x faster inference than 72B models' advantage, but no latency, throughput, hardware, or measurement protocol is provided anywhere in the manuscript. As a quantitative efficiency claim, it needs a concrete experimental setup and results; otherwise it should be removed from the abstract and contributions.
minor comments (4)
  1. [Section 5.2] Section 5.2 says results are reported as mean and standard deviation over three random seeds, but Tables 2, 3, and 5 contain only point estimates. Either include the standard deviations or remove the promise.
  2. [Section 3.1 / Appendix Table 6] Section 3.1 describes the vision encoder as CLIP-ViT-Large, but Table 6 lists vision_tower as google/siglip-so400m-patch14-384. Since the actual configuration determines the model, this discrepancy should be resolved in the main text.
  3. [Section 5.3] Section 5.3 says exact match accuracy is the principal metric for PMC-VQA, but Table 3 reports 'correctness' scores without defining how they differ from exact match. The metric should be defined precisely.
  4. [Throughout] There are several typographical and phrasing issues, including 'Evaulation Metrics' in Section 5.3, 'iamges' in the Figure 3 caption, and repeated references [31] and [32] to the same MMed-RAG paper, which should be consolidated.

Circularity Check

1 steps flagged · score 5.0 of 10

Evaluation-layer circularity: MIRA is fine-tuned on PubMedVision-curated data, then 'generalization' is claimed on PMC-VQA, which the paper itself treats as the same PubMedVision benchmark.

  1. fitted input called prediction [Section 3.3 (Reinforced CoT Generation); Section 5.5 (Results on PubMedVision, Table 3)]
    "The training dataset is derived from LLaVA-Med’s 500k alignment set, with an additional 50k high-quality CoT fine-tuning instances curated from PubMedVision and VQA-Med. ... Table 3 summarizes MIRA’s performance on the PMC-VQA dataset ... Its strong results—despite no pretraining on PMC-VQA—indicate genuine generalization."

    The CoT fine-tuning set is explicitly curated from PubMedVision, while the evaluation section is headed 'Results on PubMedVision' and reports correctness on the PMC-VQA dataset. The paper’s only defense that this is a generalization test is 'no pretraining on PMC-VQA,' but the paper itself uses PubMedVision and PMC-VQA interchangeably. Thus the PMC-VQA scores are not demonstrably independent of the fine-tuning distribution: any overlap or near-duplicate between the curated PubMedVision training instances and the PMC-VQA questions makes the 0.66/0.39 numbers a measure of memorization rather than transfer.

full rationale

Aside from the evaluation-layer overlap, MIRA’s method sections contain no formal derivation that reduces to its own inputs: the RTRA module, MRAG retrieval, and fusion losses are described as supervised engineering choices rather than as results derived from assumptions. The self-citations in the reference list (e.g., XrayGPT [28], FATE [2]) are used as baselines or unrelated prior work, not as load-bearing justification for MIRA’s design. The larger concerns—the abstract’s unqualified SOTA claim despite Table 3 trailing Qwen2.5-VL-72B and OpenAI o1, the omission of MMed-RAG/RULE from Table 2, and the inconsistency between the Section 3.3 policy-gradient equations and Section 3.4’s statement that all training uses cross-entropy loss—are correctness or reproducibility issues, not circularity. The single genuine circularity is at the evaluation stage: the model is fine-tuned on PubMedVision-derived data and then generalization is asserted on PMC-VQA, which the paper itself treats as the same benchmark, so the reported 'genuine generalization' is not an independent test.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claims rest on no new physical or conceptual entities. The free parameters are mostly standard training choices, but the retrieval counts and fusion weight directly affect the reported behavior. The most consequential assumptions are the reliability of online retrieval, the quality of the RTRA fine-tuning data, and the validity of the PMC-VQA evaluation after training on PubMedVision-derived data. The paper does not test the sensitivity of results to alpha, k_text, or k_image, and it does not describe how the 50k CoT traces were verified.

free parameters (3)
  • Modality fusion weight alpha = Not reported
    Eq. 5 in Section 3.2 defines E_final = alpha * Att(E_image) + (1-alpha) * Att(E_text); alpha in [0,1] is learned, but no value or sensitivity analysis is provided.
  • Retrieval counts k_text and k_image = k_text=3, k_image=2
    Section 3.2 and Table 6 fix the number of retrieved text chunks and images; the claimed dynamic k-selection is actually implemented as fixed top-k plus a filtering step.
  • Reward weights lambda_1, lambda_2 = Not reported
    Eq. 7 defines the reward R(z,y) = lambda_1 * FactualScore + lambda_2 * CoherenceScore; the weights are never specified and the equations are not used in the reported experiments, which use cross-entropy loss.
assumptions (3)
  • domain assumption Online web search returns relevant, reliable medical evidence.
    The MRAG component in Section 3.3 uses DuckDuckGo to fetch current text/images; the appendix shows an online retrieval from belmarrahealth.com, a consumer blog, indicating the assumption is fragile.
  • ad hoc to paper The 50k RTRA-format CoT fine-tuning instances are accurate and representative.
    Section 3.3 says the training data is 'curated from PubMedVision and VQA-Med', but the curation and quality-control process is not described; the model's behavior in the rethink loop depends on this data.
  • domain assumption PMC-VQA evaluation is a valid out-of-distribution test after training on PubMedVision-derived data.
    Used in Section 5.5; because PMC-VQA is built from PubMedVision and the fine-tuning set includes PubMedVision, this assumption is questionable and may not hold.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MIRA: A Novel Framework for Fusing Modalities in Medical RAG." pith.science (2026). https://pith.science/paper/GLCS44WQ

@misc{pith2026250707902,
  author       = {Pith},
  title        = {Pith review of: MIRA: A Novel Framework for Fusing Modalities in Medical RAG},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GLCS44WQ}},
  note         = {Machine review of arXiv:2507.07902}
}
read the original abstract

Multimodal Large Language Models (MLLMs) have significantly advanced AI-assisted medical diagnosis, but they often generate factually inconsistent responses that deviate from established medical knowledge. Retrieval-Augmented Generation (RAG) enhances factual accuracy by integrating external sources, but it presents two key challenges. First, insufficient retrieval can miss critical information, whereas excessive retrieval can introduce irrelevant or misleading content, disrupting model output. Second, even when the model initially provides correct answers, over-reliance on retrieved data can lead to factual errors. To address these issues, we introduce the Multimodal Intelligent Retrieval and Augmentation (MIRA) framework, designed to optimize factual accuracy in MLLM. MIRA consists of two key components: (1) a calibrated Rethinking and Rearrangement module that dynamically adjusts the number of retrieved contexts to manage factual risk, and (2) A medical RAG framework integrating image embeddings and a medical knowledge base with a query-rewrite module for efficient multimodal reasoning. This enables the model to effectively integrate both its inherent knowledge and external references. Our evaluation of publicly available medical VQA and report generation benchmarks demonstrates that MIRA substantially enhances factual accuracy and overall performance, achieving new state-of-the-art results. Code is released at https://github.com/mbzuai-oryx/MIRA.

Figures

Figures reproduced from arXiv: 2507.07902 by the authors.

Figure 1
Figure 1. Overview of key constraints in automatic medi [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of the MIRA (Multimodal Intelligent Retrieval and Augmentation) pipeline. The system integrates image [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Visualization of attention distribution across all [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

35 extracted references · 14 canonical work pages

  1. [1]

    Rawan AlSaad, Alaa Abd-alrazaq, Sabri Boughorbel, Arfan Ahmed, Max-Antoine Renault, Rafat Damseh, and Javaid Sheikh. 2024. Multimodal Large Language Models in Health Care: Applications, Challenges, and Future Outlook. J Med Internet Res 26 (25 Sep 2024), e59505. https://doi.org/10.2196/59505

    doi:10.2196/59505 2024

  2. [2]

    Tajamul Ashraf and Janibul Bashir. 2025. FATE: Focal-modulated Attention Encoder for Multivariate Time-series Forecasting. arXiv:2408.11336 [cs.LG] https://arxiv.org/abs/2408.11336

    work page Pith review arXiv 2025

  3. [3]

    Asma Ben Abacha, Sadid A Hasan, Vivek V Datla, Dina Demner-Fushman, and Henning Müller. 2019. Vqa-med: Overview of the medical visual question an- swering task at imageclef 2019. In Proceedings of CLEF (Conference and Labs of the Evaluation Forum) 2019 Working Notes . 9-12 September 2019

    work page 2019

  4. [4]

    Hasan, and Henning Müller

    Asma Ben Abacha, Mourad Sarrouti, Dina Demner-Fushman, Sadid A. Hasan, and Henning Müller. 2021. Overview of the VQA-Med Task at ImageCLEF 2021: Visual Question Answering and Generation in the Medical Domain. In CLEF 2021 Working Notes (CEUR Workshop Proceedings) . CEUR-WS.org, Bucharest, Romania

    work page 2021

  5. [5]

    Asma Ben Abacha, Mourad Sarrouti, Dina Demner-Fushman, Sadid A Hasan, and Henning Müller. 2021. Overview of the vqa-med task at imageclef 2021: Visual question answering and generation in the medical domain. In Proceedings of the CLEF 2021 Conference and Labs of the Evaluation Forum-working notes . 21-24 September 2021

    work page 2021

  6. [6]

    Junying Chen, Ruyi Ouyang, Anningzhe Gao, Shunian Chen, Guiming Hardy Chen, Xidong Wang, Ruifei Zhang, Zhenyang Cai, Ke Ji, Guangjun Yu, Xiang Wan, and Benyou Wang. 2024. HuatuoGPT-Vision, Towards Injecting Medical Visual Knowledge into Multimodal LLMs at Scale. arXiv:2406.19280 [cs.CV] https://arxiv.org/abs/2406.19280

    arXiv 2024

  7. [7]

    Wenhu Chen, Hexiang Hu, Xi Chen, Pat Verga, and William W. Cohen. 2022. MuRAG: Multimodal Retrieval-Augmented Generator for Open Question An- swering over Images and Text. arXiv:2210.02928 [cs.CL] https://arxiv.org/abs/ 2210.02928

    arXiv 2022

  8. [8]

    Hilmi Demirhan and Wlodek Zadrozny. 2023. Survey of Multimodal Medical Question Answering. BioMedInformatics 4, 1 (2023), 50–74

    work page 2023

Show all 35 references
  1. [9]

    Jiawen Deng, Areeba Zubair, and Ye-Jean Park. 2023. Limitations of large language models in medical applications. Postgraduate Medical Journal 99, 1178 (2023), 1298–1299

    2023

  2. [10]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human...

    2019

  3. [11]

    Shahul Es, Jithin James, Luis Espinosa-Anke, and Steven Schockaert

  4. [12]

    Yu Gu, Robert Tinn, Hao Cheng, Michael Lucas, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, and Hoifung Poon. 2021. Domain-specific language model pretraining for biomedical natural language processing. ACM Transactions on Computing for Healthcare (HEALTH) 3, 1 (...

    2021

  5. [13]

    Paul Hager, Friederike Jungmann, Robbie Holland, Kunal Bhagat, Inga Hubrecht, Manuel Knauer, Jakob Vielhauer, Marcus Makowski, Rickmer Braren, Georgios Kaissis, et al. 2024. Evaluation and mitigation of the limitations of large language models in clinical decision-making. Natu...

    2024

  6. [14]

    Alistair EW Johnson, Tom J Pollard, Seth J Berkowitz, Nathaniel R Greenbaum, Matthew P Lungren, Chih-ying Deng, Roger G Mark, and Steven Horng. 2019. MIMIC-CXR, a de-identified publicly available database of chest radiographs with free-text reports. Scientific data 6, 1 (2019), 317

    2019

  7. [15]

    Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense Passage Retrieval for Open- Domain Question Answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP...

    2020

  8. [16]

    Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Naumann, Hoifung Poon, and Jianfeng Gao. 2023. Llava-med: Training a large language-and-vision assistant for biomedicine in one day. arXiv preprint arXiv:2306.00890 (2023)

    2023 arXiv

  9. [17]

    Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Naumann, Hoifung Poon, and Jianfeng Gao. 2024. Llava-med: Train- ing a large language-and-vision assistant for biomedicine in one day. Advances in Neural Information Processing Systems 36 (2024)

    2024

  10. [18]

    Weixiong Lin, Ziheng Zhao, Xiaoman Zhang, Chaoyi Wu, Ya Zhang, Yanfeng Wang, and Weidi Xie. 2023. PMC-CLIP: Contrastive Language-Image Pre-training using Biomedical Documents. Medical Image Computing and Computer Assisted Intervention

    2023

  11. [19]

    Zhihong Lin, Donghao Zhang, Qingyi Tac, Danli Shi, Gholamreza Haffari, Qi Wu, Mingguang He, and Zongyuan Ge. 2022. Medical Visual Question Answering: A Survey. arXiv preprint arXiv:2111.10056 (2022)

    2022 arXiv

  12. [20]

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2023. Improved Baselines with Visual Instruction Tuning

    2023

  13. [21]

    Yunyi Liu, Zhanyu Wang, Dong Xu, and Luping Zhou. 2023. Q2ATransformer: Improving Medical VQA via an Answer Querying Decoder. arXiv preprint arXiv:2304.01611 (2023)

    2023 arXiv

  14. [22]

    Michael Moor, Qian Huang, Shirley Wu, Michihiro Yasunaga, Yash Dalmia, Jure Leskovec, Cyril Zakka, Eduardo Pontes Reis, and Pranav Rajpurkar. 2023. Med- flamingo: a multimodal medical few-shot learner. In Machine Learning for Health (ML4H). PMLR, 353–367

    2023

  15. [23]

    Zabir Al Nazi and Wei Peng. 2024. Large language models in healthcare and medical domain: A review. In Informatics, Vol. 11. MDPI, 57

    2024

  16. [24]

    Jiwoo Park, Kangrok Oh, Kyunghwa Han, and Young Han Lee. 2024. Patient- centered radiology reports with generative artificial intelligence: adding value to radiology reporting. Scientific Reports 14, 1 (2024), 13218

    2024

  17. [25]

    Cheng Peng, Xi Yang, Aokun Chen, Kaleb E Smith, Nima PourNejatian, Anthony B Costa, Cheryl Martin, Mona G Flores, Ying Zhang, Tanja Magoc, et al. 2023. A Study of Generative Large Language Model for Medical Research and Healthcare. arXiv preprint arXiv:2305.13523 (2023)

    2023 arXiv

  18. [26]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. arXiv:2103.000...

    2021 arXiv

  19. [27]

    Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. 2025. Kimi k1. 5: Scaling reinforcement learning with llms. arXiv preprint arXiv:2501.12599 (2025)

    2025 arXiv

  20. [28]

    Omkar Thawkar, Abdelrahman Shaker, Sahal Shaji Mullappilly, Hisham Cholakkal, Rao Muhammad Anwer, Salman Khan, Jorma Laaksonen, and Fa- had Shahbaz Khan. 2023. Xraygpt: Chest radiographs summarization using medical vision-language models. arXiv preprint arXiv:2306.07971 (2023)

    2023 arXiv

  21. [29]

    Arun James Thirunavukarasu, Darren Shu Jeng Ting, Kabilan Elangovan, Laura Gutierrez, Ting Fang Tan, and Daniel Shu Wei Ting. 2023. Large language models in medicine. Nature Medicine 29 (2023), 1930–1940. https://api.semanticscholar. org/CorpusID:259947046 Conference, paper, A...

    2023

  22. [30]

    Haochun Wang, Chi Liu, Nuwa Xi, Zewen Qiang, Sendong Zhao, Bing Qin, and Ting Liu. 2023. HuaTuo: Tuning LLaMA Model with Chinese Medical Knowledge. arXiv preprint arXiv:2304.06975 (2023)

    2023 arXiv

  23. [32]

    Peng Xia, Kangyu Zhu, Haoran Li, Tianze Wang, Weijia Shi, Sheng Wang, Linjun Zhang, James Zou, and Huaxiu Yao. 2024. MMed-RAG: Versatile Multimodal RAG System for Medical Vision Language Models. arXiv:2410.13085 [cs.LG] https://arxiv.org/abs/2410.13085

    2024 arXiv

  24. [34]

    Peng Xia, Kangyu Zhu, Haoran Li, Hongtu Zhu, Yun Li, Gang Li, Linjun Zhang, and Huaxiu Yao. 2024. Rule: Reliable multimodal rag for factuality in medical vision language models. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing . 1081–1093

    2024

  25. [35]

    Davison, Quanzheng Li, Yong Chen, Hongfang Liu, and Lichao Sun

    Kai Zhang, Jun Yu, Zhiling Yan, Yixin Liu, Eashan Adhikarla, Sunyang Fu, Xun Chen, Chen Chen, Yuyin Zhou, Xiang Li, Lifang He, Brian D. Davison, Quanzheng Li, Yong Chen, Hongfang Liu, and Lichao Sun. 2023. BiomedGPT: A Unified and Generalist Biomedical Generative Pre-trained T...

    2023 arXiv

  26. [36]

    Sheng Zhang, Yanbo Xu, Naoto Usuyama, Hanwen Xu, Jaspreet Bagga, Robert Tinn, Sam Preston, Rajesh Rao, Mu Wei, Naveen Valluri, et al . 2023. Biomed- CLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs. arXiv preprint arXiv...

    2023 arXiv

  27. [2023]

    arXiv:2309.15217 [cs.CL] https://arxiv.org/abs/2309.15217

    RAGAS: Automated Evaluation of Retrieval Augmented Generation. arXiv:2309.15217 [cs.CL] https://arxiv.org/abs/2309.15217

    arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.