REVIEW 5 major objections 7 minor 3 cited by
MMedPO: Aligning Medical Vision-Language Models with Clinical-Aware Multimodal Preference Optimization
T0 review · 5 major / 7 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read MMedPO claims clinical-relevance weighting of preference data improves medical VQA accuracy by 14.2% and report generation by 51.7% over prior methods.
desk verdict The weighting idea is worth a look, but the lesion-noising stage as written compares the same ground-truth text on clean vs noisy images, so the paper's main mechanism is not what it claims. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is a weighted direct preference optimization (DPO) loss, Eq. (3), in which each preference tuple $(x, x^*, y_w, y_l, s')$ contributes proportionally to its normalized clinical-relevance weight $s'$: $$\mathcal{L}_{\mathrm{MMedPO}} = -\mathbb{E}_{(x,x^*,y_w,y_l,s')\sim\mathcal{D}_o}\left[s' \log\$\sigma$\left(\$\alpha$ \log\frac{\pi_\$\theta$(y_w|x)}{\pi_o(y_w|x)} - \$\alpha$ \log\frac{\pi_\$\theta$(y_l|x^*)}{\pi_o(y_l|x^*)}\right)\right].$$ The weight $s'$ comes from one of two sources: a multi-agent Med-LLM consensus score for hallucinated-text dispreferred responses, or the visual tool's lesion-detection confidence for image-noise pairs, and is z-score normalized and clipped to $[\alpha,\beta]=[0.75,1.25]$ before use. The noised image in the second preference source is built by Eq. (2): $$x_v^* = \sqrt{\bar{\xi}_k}(x_v\odot h) + \sqrt{1-\bar{\xi}_k}(\epsilon\odot h) + (x_v\odot(1-h)),$$ where $h$ is the lesion-region heatmap from the visual tool, so only the detected disease area is perturbed and the rest of the image is left intact.
What would settle it
Measure the Dice overlap between the visual tool's 'disease' heatmaps and expert lesion segmentations on the four datasets, then compare MMedPO's margin over the global-noise ablation across images grouped by that overlap; if the margin disappears on low-overlap images, the localized-lesion mechanism is not doing the work attributed to it.
Extended reading notes
Core claim
On the paper's own terms, the central discovery is that preference optimization for Med-LVLMs becomes more effective when the training signal is clinical rather than generic. The authors construct dispreference in two complementary ways: hallucinated medical answers selected or generated through GPT-4o, and lesion-region neglect produced by locally noising the heatmap-detected disease area. They then weigh each preference pair by a quantified clinical relevance score—multi-agent Med-LLM consensus for text responses, visual-tool confidence for lesion localization—and run a normalized weighted DPO. The reported result is that this pipeline outperforms DPO, POVID, FiSAO, SIMA, STLLaVA-Med, and self-rewarding baselines on all four datasets, with the largest margins on open-ended and report-generation tasks.
Load-bearing premise
The training signal from lesion-localized noise is only as good as the visual tool's disease heatmaps: if the tool misses or misplaces the lesion, the noised image teaches the model to ignore the true diagnostic region, and the confidence-score weights may reward that mislocalization.
Editorial extensions
If this is right
- Weighting preference samples by clinical relevance is a model-agnostic upgrade: MMedPO improves both LLaVA-Med-1.5 and LLaVA-Med++ without changing the backbone.
- Localized lesion-noising is the stronger ingredient on report generation, so datasets dominated by free-form image-to-text pairs benefit most from visual-tool-guided dispreference.
- Clinical-relevance weights provide a clear gain over unweighted DPO, with an average 2.3% improvement on VQA and 18.5% on report generation in the ablation study.
- Multi-agent consensus scoring beats single Med-LLM scoring by an average of 3.6%, indicating that score reliability directly affects alignment quality.
- Combining hallucinated-text pairs and lesion-region pairs outperforms either preference source alone, so the two mechanisms are complementary.
Reading between the lines
- The z-score normalization with fixed clipping bounds $[0.75,1.25]$ is a free hyperparameter; testing wider or dataset-dependent bounds would reveal how sensitive the reported gains are to the weighting schedule.
- The clinical scorer sees text only for hallucinated-text samples; integrating the lesion heatmaps into that score could catch cases where a plausible answer contradicts the image, which the current pipeline may miss.
- If the lesion-localization mechanism is what carries the report-generation gain, the same recipe should transfer to CT, ultrasound, or pathology slides whenever a heatmap tool exists; a rerun on such data would directly test that transfer.
- The 51.7% report-generation improvement is measured with lexical metrics (BLEU/ROUGE/METEOR); a structured clinical-evaluation study, such as checking whether generated reports match expert-labeled findings, would show whether the gain translates to clinically meaningful text.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. MMedPO proposes a preference-optimization method for medical vision-language models that (i) constructs dispreferred responses by injecting plausible hallucinations and by locally noising predicted lesion regions, and (ii) weights DPO samples by a clinical relevance score obtained from a multi-Med-LLM debate and from visual-tool confidence. The method is evaluated on four public datasets (SLAKE, VQA-RAD, IU-Xray, MIMIC-CXR) with LLaVA-Med-1.5 and LLaVA-Med++ backbones, comparing against DPO, POVID, FiSAO, SIMA, STLLaVA-Med, and a self-rewarding baseline. The paper reports consistent gains over these baselines and includes ablations for the two data-curation strategies, the clinical-relevance weighting, local versus global noise, and single versus multiple Med-LLM scoring.
Significance. If the central claim held, MMedPO would offer a practical recipe for making preference data more clinically meaningful in medical VLMs, and the reported gains on external benchmarks would be a useful advance. The paper has strengths: it uses standard public benchmarks, compares with several preference-optimization baselines, provides ablations, reports compatibility with a second backbone, and releases code. However, the central mechanism for the lesion-region-neglect preference pairs is not implemented as described, the abstract's headline numbers do not match the body, and several evidence claims are weaker than stated. These issues are load-bearing for the paper's main contribution, so the current version cannot be accepted without substantial revision.
major comments (5)
- [§3.1, Eq. (2)–(3), Algorithm 1] For the Dv preference pairs, the preferred and dispreferred responses are the same ground-truth y: the text says 'the original image xv paired with the ground truth y is considered preferred, while the image with localized noise xk paired with the same ground truth y is regarded as dispreferred,' and Algorithm 1 stores {xv, x∗_v, y, sv}. Substituting yw = yl = y into Eq. (3) gives a DPO loss that contrasts log πθ(y|x) with log πθ(y|x∗) for the same response, i.e., a likelihood ratio between clean and locally noised inputs, not a preference between a correct response and a response that neglects lesion regions. The manuscript never constructs the claimed 'responses that overlook critical regions.' Since Table 8 attributes most of the report-generation gain to Stage 2, the explanation for the headline improvement is not supported by the construction actually specified. The authors should either generate yl as the target model's output on the noised image (a genuine neglect response) and use it in Eq. (3), or explicitly reframe Stage 2 as a robustness regularizer and provide corresponding evidence.
- [Abstract and §4.2] The abstract's claim of 'averaging 14.2% and 51.7% across the Med-VQA and report generation tasks' is not consistent with the body. Section 4.2 reports average improvements over the best baseline of 15.8% and 10.3% for open-ended and closed-ended VQA questions, and 61.9% and 26.0% for IU-Xray and MIMIC-CXR report generation. The 14.2% figure appears in the text as the average improvement over the SFT baseline across all four datasets, not as a Med-VQA average. These numbers must be reconciled in a single consistent presentation.
- [§4.1 and Conclusion] The paper repeatedly characterizes BLEU, ROUGE-L, and METEOR as measures of 'factual accuracy' for report generation. These are lexical-overlap metrics and do not measure factual correctness or clinical validity; a report can score high while containing clinically wrong statements, and vice versa. The abstract and conclusion accordingly overstate what the experiments demonstrate about factuality. The authors should either use clinically meaningful factuality metrics (e.g., radiology-specific factual error rates or expert evaluation) or restrict their claims to what the metrics support.
- [Tables 1–4, 7–9] No error bars, standard deviations, or significance tests are reported for any of the experimental results. Several differences in Table 1 are small (for example, MIMIC-CXR ROUGE-L gains of about 1 point over DPO or FiSAO), so without repeated runs or statistical testing it is unclear which improvements are reliable. At minimum, the authors should report multiple seeds with variance and, where possible, significance tests for the main comparisons.
- [§4.4.1 and §3.2.1] The claim that MMedPO improves the 'clinical relevance' of responses (Figure 6) is evaluated by Med-LLM judges of the same type used in the multi-agent collaboration that produces the training weights. This supporting evidence is not independent of the training procedure, so it cannot be used as confirmatory evidence for the clinical-relevance improvement. The authors should either use held-out human expert evaluation or a distinct evaluator model not involved in training, and should report agreement with human judgments if Med-LLM scores are used.
minor comments (7)
- [§3.3] Eq. (3) uses πo to denote the reference policy, whereas Eq. (1) uses πref; please unify the notation.
- [§3.2.2] There are typos and grammar issues: 'Quatify' instead of 'Quantify' in Algorithm 1, 'importance' instead of 'important' in Section 3.2.2, and the sentence 'These scores, serve as weights' has an extraneous comma.
- [Table 1 caption] The caption says the table covers SLAKE, VQA-RAD, and IU-Xray, but the table also reports MIMIC-CXR; please include all four datasets in the caption.
- [§3.1, Eq. (2)] The localization accuracy of the MedKLIP heatmaps h = T(xv) is never measured on the four datasets. Since the localized noise is applied exactly where the heatmap is nonzero, reporting a quantitative validation of the heatmaps (e.g., overlap with ground-truth lesion annotations, or a manual audit) would strengthen the interpretation of the Stage 2 experiments.
- [Appendix B] The hyperparameters α, β, μ, σ², ξ_k, k, the number of debate rounds r, and the choice of Med-LLM judges are fixed values without sensitivity analysis. The claim that all hyperparameters are kept consistent does not address whether results depend strongly on these choices.
- [Figure 3 and Table 2] The figures and tables report an 'average score on each dataset' without defining how the average is computed across the different metrics for each dataset; please specify the aggregation formula.
- [§4.3.4] The sentence 'As illustrated in Table 4' appears to refer to the compatibility analysis, but the compatibility results are shown in Figure 4; please correct the cross-reference.
Circularity Check
D_v preference pairs are self-definitional (same y as y_w and y_l), so Eq. (3) reduces to a clean-vs-noised likelihood ratio; clinical-relevance support reuses the same Med-LLM oracle that creates the training weights.
-
self definitional
[Section 3.1 (Dv construction, Eq. 2), Algorithm 1 lines 11-12, Eq. (3)]
"In this approach, the original image xv paired with the ground truth y is considered preferred, while the image with localized noise xk paired with the same ground truth y is regarded as dispreferred. The preference data constructed using this strategy is denoted as Dv."
Algorithm 1 stores {x_v, x*_v, y, s_v} for every D_v sample, so y_w = y_l = y. Substituting into Eq. (3) gives log σ(α log πθ(y|x)/πref(y|x) − α log πθ(y|x*)/πref(y|x*)): the contrast is between clean and noised images for one and the same response, not between a correct response and a response that overlooks lesion regions. The stated mechanism—"dispreferred responses that overlook critical regions"—is therefore not present in the loss; the construction reduces by definition to an image-robustness regularizer. Since Table 8 attributes most report-generation gain to this stage, the paper's explanation of its improvement is not supported by the preference pair it actually constructs.
-
other
[Section 4.4.1 (Figure 6) vs Section 3.2.1 and Eq. (3)]
"The evaluation of response using clinical relevance from Med-LLMs quantitatively shows that MMedPO consistently achieves significantly higher clinical relevance scores."
The sample weights in Eq. (3) are derived from clinical-relevance scores produced by a multi-agent Med-LLM system (Section 3.2.1, with LLaMA3-Med42 and BioMistral listed in Implementation Details). The supporting claim that MMedPO improves clinical relevance is then measured with the same construct—clinical relevance judged by Med-LLMs—in Figure 6. The evaluation is not independent of the training signal: the same oracle family defines both what is weighted during optimization and what counts as improvement. This is secondary to the external factual-accuracy benchmarks, but it makes the clinical-relevance evidence partially self-referential.
full rationale
MMedPO's headline factual-accuracy gains are measured on external datasets (VQA-RAD, SLAKE, MIMIC-CXR, IU-Xray) against standard preference-optimization baselines, so the central empirical claim is not circular and could in principle fail. No load-bearing self-citation chain or imported uniqueness theorem is present. However, two steps are partially self-referential. First, the lesion-neglect preference pairs in D_v are defined with y_w = y_l = y; Eq. (3) then contrasts the same response under clean versus noised images rather than a correct response against a lesion-neglecting response. The claimed "dispreferred responses that overlook critical regions" are never constructed, so the report-generation gains attributed to Stage 2 are explained by a mechanism the loss does not contain. This is a construction-level self-definitional reduction. Second, the "improved clinical relevance" evidence in Figure 6 uses Med-LLM clinical-relevance scoring, the same construct that supplies the sample weights in Eq. (3); this supporting claim is not independently grounded. Because the main factual-accuracy benchmark results remain external and testable, the circularity is partial rather than total.
Assumptions & free parameters
free parameters (5)
- Score normalization bounds alpha and beta =
alpha=0.75, beta=1.25
- Normalization moment hyperparameters mu and sigma squared =
mu=1, sigma^2=0.1
- Noise schedule hyperparameters xi_k and noise step k =
not specified
- Number of multi-agent debate rounds r =
5
- Number and choice of Med-LLM judges =
3 agents: LLaMA3-Med42-7B, LLaMA3-Med42-70B, BioMistral-7B
assumptions (5)
- domain assumption Weighted DPO with clinical relevance weights improves modality alignment and reduces hallucinations in Med-LVLMs.
- domain assumption Clinical relevance of dispreferred text can be reliably assessed by Med-LLMs without seeing the image.
- domain assumption MedKLIP's disease heatmaps accurately localize diagnostically relevant lesion regions.
- ad hoc to paper Score normalization by mean/variance and clipping preserves relative sample importance.
- standard math The Bradley-Terry preference model underlying DPO extends to image-conditioned medical responses.
invented entities (1)
-
Clinical relevance score used as per-sample weight
Cite this review
Pith. "Pith review of MMedPO: Aligning Medical Vision-Language Models with Clinical-Aware Multimodal Preference Optimization." pith.science (2026). https://pith.science/paper/ZVNQROOR
@misc{pith2026241206141,
author = {Pith},
title = {Pith review of: MMedPO: Aligning Medical Vision-Language Models with Clinical-Aware Multimodal Preference Optimization},
year = {2026},
howpublished = {\url{https://pith.science/paper/ZVNQROOR}},
note = {Machine review of arXiv:2412.06141}
}
read the original abstract
The advancement of Large Vision-Language Models (LVLMs) has propelled their application in the medical field. However, Medical LVLMs (Med-LVLMs) encounter factuality challenges due to modality misalignment, where the models prioritize textual knowledge over visual input, leading to hallucinations that contradict information in medical images. Previous attempts to enhance modality alignment in Med-LVLMs through preference optimization have inadequately mitigated clinical relevance in preference data, making these samples easily distinguishable and reducing alignment effectiveness. To address this challenge, we propose MMedPO, a novel multimodal medical preference optimization approach that considers the clinical relevance of preference samples to enhance Med-LVLM alignment. MMedPO curates multimodal preference data by introducing two types of dispreference: (1) plausible hallucinations injected through target Med-LVLMs or GPT-4o to produce medically inaccurate responses, and (2) lesion region neglect achieved through local lesion-noising, disrupting visual understanding of critical areas. We then calculate clinical relevance for each sample based on scores from multiple Med-LLMs and visual tools, and integrate these scores into the preference optimization process as weights, enabling effective alignment. Our experiments demonstrate that MMedPO significantly enhances factual accuracy in Med-LVLMs, achieving substantial improvements over existing preference optimization methods by averaging 14.2% and 51.7% across the Med-VQA and report generation tasks. Our code are available in https://github.com/aiming-lab/MMedPO.
Figures
Figures from the paper (7 more)
Forward citations
Cited by 3 Pith papers
-
CheXPO: Preference Optimization for Chest X-ray VLMs with Counterfactual Rationale
A preference optimization strategy using confidence-based hard example mining, similarity retrieval, and synthetic counterfactual rationales improves chest X-ray VQA accuracy by 8.93% relative over supervised fine-tuning.
-
HSCR: Hierarchical Self-Contrastive Rewarding for Aligning Medical Vision Language Models
HSCR uses visual token dropout and logit contrast to construct self-generated dispreferred answers, then trains a medical VLM with explicit and implicit preference losses, improving zero-shot Rad-VQA, SLAKE, and PathV...
-
Improving Medical Large Vision-Language Models with Abnormal-Aware Feedback
UMed-LVLM uses GPT-4V-generated abnormality data and abnormal-aware rewards to improve medical image diagnosis and localization.
Reference graph
Works this paper leans on
-
[5]
Holistic analysis of hallucination in gpt-4v (ision): Bias and interference challenges
Cui, C., Zhou, Y ., Yang, X., Wu, S., Zhang, L., Zou, J., and Yao, H. Holistic analysis of hallucination in gpt-4v (ision): Bias and interference challenges. arXiv preprint arXiv:2311.03287,
-
[6]
Fine-grained verifiers: Preference modeling as next-token prediction in vision-language alignment
Cui, C., Zhang, A., Zhou, Y ., Chen, Z., Deng, G., Yao, H., and Chua, T.-S. Fine-grained verifiers: Preference modeling as next-token prediction in vision-language alignment. arXiv preprint arXiv:2410.14148,
-
[8]
Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783,
-
[9]
Mdocagent: A multi-modal multi-agent framework for document understanding
Han, S., Xia, P., Zhang, R., Sun, T., Li, Y ., Zhu, H., and Yao, H. Mdocagent: A multi-modal multi-agent framework for document understanding. arXiv preprint arXiv:2503.13964,
-
[11]
J., Shen, Y ., Wallis, P., Allen-Zhu, Z., Li, Y ., Wang, S., Wang, L., and Chen, W
Hu, E. J., Shen, Y ., Wallis, P., Allen-Zhu, Z., Li, Y ., Wang, S., Wang, L., and Chen, W. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685,
-
[12]
Ophnet: A large- scale video benchmark for ophthalmic surgical workflow understanding
Hu, M., Xia, P., Wang, L., Yan, S., Tang, F., Xu, Z., Luo, Y ., Song, K., Leitner, J., Cheng, X., et al. Ophnet: A large- scale video benchmark for ophthalmic surgical workflow understanding. arXiv preprint arXiv:2406.07471,
-
[13]
Jiang, Y ., Chen, J., Yang, D., Li, M., Wang, S., Wu, T., Li, K., and Zhang, L. Medthink: Inducing medical large- scale visual language models to hallucinate less by think- ing more. arXiv preprint arXiv:2406.11451,
-
[15]
Crafting papers on machine learning
Langley, P. Crafting papers on machine learning. In Langley, P. (ed.),Proceedings of the 17th International Conference on Machine Learning (ICML 2000), pp. 1207–1216, Stan- ford, CA,
2000
Show all 33 references
-
[16]
Llava-med: Training a large language-and-vision assistant for biomedicine in one day
Li, C., Wong, C., Zhang, S., Usuyama, N., Liu, H., Yang, J., Naumann, T., Poon, H., and Gao, J. Llava-med: Training a large language-and-vision assistant for biomedicine in one day. In Thirty-seventh Conference on Neural Informa- tion Processing Systems Datasets and Benchmarks...
-
[17]
A comprehensive study of gpt-4v’s multimodal capabilities in medical imaging
Li, Y ., Liu, Y ., Wang, Z., Liang, X., Liu, L., Wang, L., Cui, L., Tu, Z., Wang, L., and Zhou, L. A comprehensive study of gpt-4v’s multimodal capabilities in medical imaging. arXiv preprint arXiv:2310.20381, 2023b. Lin, C.-Y . Rouge: A package for automatic evaluation of sum...
-
[19]
org/abs/2303.08774
https://arxiv. org/abs/2303.08774. Papineni, K., Roukos, S., Ward, T., and Zhu, W.-J. Bleu: a method for automatic evaluation of machine transla- tion. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pp. 311–318,
-
[20]
Multimedeval: A benchmark and a toolkit for evaluating medical vision- language models
Royer, C., Menze, B., and Sekuboyina, A. Multimedeval: A benchmark and a toolkit for evaluating medical vision- language models. arXiv preprint arXiv:2402.09262 ,
-
[21]
Conflictbank: A benchmark for evaluating the influence of knowledge conflicts in llm
Su, Z., Zhang, J., Qu, X., Zhu, T., Li, Y ., Sun, J., Li, J., Zhang, M., and Cheng, Y . Conflictbank: A benchmark for evaluating the influence of knowledge conflicts in llm. arXiv preprint arXiv:2408.12076, 2024a. 10 MMedPO: Aligning Medical Vision-Language Models with Clinica...
-
[23]
J., Ting, D
Thirunavukarasu, A. J., Ting, D. S. J., Elangovan, K., Gutier- rez, L., Tan, T. F., and Ting, D. S. W. Large language models in medicine. Nature medicine, 29(8):1930–1940,
1930
-
[24]
Mj-video: Fine- grained benchmarking and rewarding video preferences in video generation
Tong, H., Wang, Z., Chen, Z., Ji, H., Qiu, S., Han, S., Geng, K., Xue, Z., Zhou, Y ., Xia, P., et al. Mj-video: Fine- grained benchmarking and rewarding video preferences in video generation. arXiv preprint arXiv:2502.01719,
-
[26]
Can gpt- 4v (ision) serve medical applications? case studies on gpt-4v for multimodal medical diagnosis
Wu, C., Lei, J., Zheng, Q., Zhao, W., Lin, W., Zhang, X., Zhou, X., Zhao, Z., Zhang, Y ., Wang, Y ., et al. Can gpt- 4v (ision) serve medical applications? case studies on gpt-4v for multimodal medical diagnosis. arXiv preprint arXiv:2310.09909, 2023a. Wu, C., Zhang, X., Zhang...
-
[27]
Medtrinity-25m: A large-scale multimodal dataset with multigranular anno- tations for medicine
Xie, Y ., Zhou, C., Gao, L., Wu, J., Li, X., Zhou, H.-Y ., Liu, S., Xing, L., Zou, J., Xie, C., et al. Medtrinity-25m: A large-scale multimodal dataset with multigranular anno- tations for medicine. arXiv preprint arXiv:2408.02900,
-
[28]
Pmc-vqa: Visual instruction tuning for medical visual question answering
Zhang, X., Wu, C., Zhao, Z., Lin, W., Zhang, Y ., Wang, Y ., and Xie, W. Pmc-vqa: Visual instruction tuning for medical visual question answering. arXiv preprint arXiv:2305.10415,
-
[29]
Beyond hallucinations: Enhancing lvlms through hallucination-aware direct preference optimization
Zhao, Z., Wang, B., Ouyang, L., Dong, X., Wang, J., and He, C. Beyond hallucinations: Enhancing lvlms through hallucination-aware direct preference optimization. arXiv preprint arXiv:2311.16839,
-
[30]
Align- ing modalities in vision large language models via pref- erence fine-tuning
Zhou, Y ., Cui, C., Rafailov, R., Finn, C., and Yao, H. Align- ing modalities in vision large language models via pref- erence fine-tuning. arXiv preprint arXiv:2402.11411 , 2024a. Zhou, Y ., Fan, Z., Cheng, D., Yang, S., Chen, Z., Cui, C., Wang, X., Li, Y ., Zhang, L., and Ya...
-
[31]
Minigpt-4: Enhancing vision-language understanding with advanced large language models
Zhu, D., Chen, J., Shen, X., Li, X., and Elhoseiny, M. Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304.10592,
-
[32]
Train (visual)
Data statistics for the training set of four datasets under two different task settings. “Train (visual)” refers to the number of visual-only preference data, while “Train (text)” indicates the number of text-only preference data. Dataset Train (visual) Train (text) Train (all...
-
[33]
#Images, #QA items and #Reports mean the number of images, QA pairs and reports, respectively
Data statistics of test set. #Images, #QA items and #Reports mean the number of images, QA pairs and reports, respectively. Dataset #Images #QA items #Reports IU-Xray 590 - 590 MIMIC-CXR 200 - 200 SLAKE 641 1061 - VQA-RAD 315 451 - A.2. Involved Datasets We leverage four open-...
2020
-
[2004]
Slake: A semantically-labeled knowledge-enhanced dataset for medical visual question answering
Liu, B., Zhan, L.-M., Xu, L., Ma, L., Yang, Y ., and Wu, X.- M. Slake: A semantically-labeled knowledge-enhanced dataset for medical visual question answering. In 2021 IEEE 18th International Symposium on Biomedical Imag- ing (ISBI), pp. 1650–1654. IEEE,
2021
-
[2005]
Chateval: Towards better llm-based evaluators through multi-agent debate
Chan, C.-M., Chen, W., Su, Y ., Yu, J., Xue, W., Zhang, S., Fu, J., and Liu, Z. Chateval: Towards better llm-based evaluators through multi-agent debate. In The Twelfth International Conference on Learning Representations. Chen, J., Yang, D., Wu, T., Jiang, Y ., Hou, X., Li, M...
-
[2016]
Enhancing large vision language models with self-training on image comprehension.arXiv preprint arXiv:2405.19716,
Deng, Y ., Lu, P., Yin, F., Hu, Z., Shen, S., Zou, J., Chang, K.-W., and Wang, W. Enhancing large vision language models with self-training on image comprehension.arXiv preprint arXiv:2405.19716,
-
[2019]
Y ., Xu, N., Zhang, S., Poon, H., and Chen, M
Wang, F., Zhou, W., Huang, J. Y ., Xu, N., Zhang, S., Poon, H., and Chen, M. mdpo: Conditional preference opti- mization for multimodal large language models. arXiv preprint arXiv:2406.11839, 2024a. Wang, X., Chen, J., Wang, Z., Zhou, Y ., Zhou, Y ., Yao, H., Zhou, T., Goldste...
-
[2020]
Biomistral: A collection of open-source pretrained large language models for medi- cal domains
Labrak, Y ., Bazoge, A., Morin, E., Gourraud, P.-A., Rou- vier, M., and Dufour, R. Biomistral: A collection of open-source pretrained large language models for medi- cal domains. arXiv preprint arXiv:2402.10373,
-
[2021]
S., Cholakkal, H., Anwer, R
Thawkar, O., Shaker, A., Mullappilly, S. S., Cholakkal, H., Anwer, R. M., Khan, S., Laaksonen, J., and Khan, F. S. Xraygpt: Chest radiographs summarization us- ing medical vision-language models. arXiv preprint arXiv:2306.07971,
-
[2022]
Direct preference optimization for suppressing hallucinated prior exams in radiology report generation
Banerjee, O., Zhou, H.-Y ., Adithan, S., Kwak, S., Wu, K., and Rajpurkar, P. Direct preference optimization for suppressing hallucinated prior exams in radiology report generation. arXiv preprint arXiv:2406.06496,
-
[2023]
Training a helpful and harmless assistant with rein- forcement learning from human feedback
Bai, Y ., Jones, A., Ndousse, K., Askell, A., Chen, A., Das- Sarma, N., Drain, D., Fort, S., Ganguli, D., and Henighan, T. Training a helpful and harmless assistant with rein- forcement learning from human feedback. arXiv preprint arXiv:2204.05862,
-
[2024]
K., Raha, T., Khan, S., and Pimentel, M
Christophe, C., Kanithi, P. K., Raha, T., Khan, S., and Pimentel, M. A. Med42-v2: A suite of clinical llms. arXiv preprint arXiv:2408.06142,
-
[2025]
P., Michalson, A
Hein, D., Chen, Z., Ostmeier, S., Xu, J., Varma, M., Reis, E. P., Michalson, A. E., Bluethgen, C., Shin, H. J., Lan- glotz, C., et al. Preference fine-tuning for factuality in 9 MMedPO: Aligning Medical Vision-Language Models with Clinical-Aware Multimodal Preference Optimizat...
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.