{"id":"88ed53c2-4159-41a2-a4aa-a7eea7def40c","arxiv_id":"2412.06141","paper_version":4,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"MMedPO weights preference-optimization training samples by clinical relevance scores, combining hallucinated text answers and locally noised lesion images, and reports improved medical VQA and report generation metrics.","lead":"This paper proposes MMedPO, a training method that adds clinically relevant bad examples to preference data and weights them by how clinically meaningful they are, to reduce hallucinations in medical vision-language models. It reports accuracy gains on medical visual question answering and radiology report generation across four datasets.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Lesion-localized dispreference pairs in §3.1 use the same ground-truth response as both yw and yl, so the DPO loss never contrasts a correct against an incorrect response; the claimed 'responses that overlook critical regions' are not actually constructed.","rationale":"The reader's weakest assumption concerned MedKLIP accuracy of lesion masks. I agree that is a risk, but a more basic issue precedes it: the Dv preference pairs are not preference pairs at all under the paper's own definition, because yl = yw. Even a perfect lesion detector would not fix this. The DPO loss in Eq. 3 is well-defined mathematically, but its semantics differ from the paper's claim that MMedPO 'provokes dispreference by neglecting lesion regions'. The model is never shown an incorrect answer attributed to lesion neglect; it is shown the same correct answer paired with a corrupted image. Consequently, the mechanism advanced to explain the large Stage 2 gains in Table 8 is absent from the actual construction. This is an internal inconsistency, not a disagreement with external consensus. It is addressable: the authors can either correct the construction (e.g., generate yl on noised images) or revise the claim. Because the empirical tables may still be reproducible, CONDITIONAL remains the right verdict; the paper should not be accepted until this is resolved. The reader and I partially overlap: both concerns target Dv, but the reader assumes the pair structure is valid and questions the heatmap, while I question the pair structure itself.","tokens_in":18055,"tokens_out":8435,"duration_ms":79714,"concrete_test":"Inspect the released preference data: for Dv samples, is yl identical to y? If yes, retrain Stage 2 with yl replaced by the target Med-LVLM's own response to x*_v (the actual lesion-neglect response), keeping all other settings fixed. Compare Table 8 Stage 2 numbers; if the corrected construction does not reproduce the reported gains, the current results do not validate the claimed lesion-region mechanism.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.1 defines two preference strategies. For Dv (Eq. 2), the original image plus ground truth y is preferred, while the locally noised image plus 'the same ground truth y' is dispreferred. Algorithm 1 (lines 11–12) stores {xv, x*_v, y, sv}. Thus for every Dv sample yw = yl = y. Substituting into the DPO loss (Eq. 3) gives log σ(α log πθ(y|x)/πref(y|x) − α log πθ(y|x*)/πref(y|x*)): a likelihood ratio between two inputs for the same response, not a preference between two responses for the same input. The dispreferred 'response' is textually the correct answer; it cannot 'neglect lesion regions'. The model is being trained to assign higher likelihood to y on clean images than on noisy images, which is a robustness regularizer, not the lesion-neglect teaching described in §1 and §3.1. Since Table 8 attributes most of the report-generation gain to Stage 2, the paper's explanation for its headline improvement is not supported by the construction it specifies. If the authors instead intended yl to be the target model's output on the noised image (a genuine neglect response), the text and Algorithm 1 do not say so.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"MMedPO proposes a preference-optimization method for medical vision-language models that (i) constructs dispreferred responses by injecting plausible hallucinations and by locally noising predicted lesion regions, and (ii) weights DPO samples by a clinical relevance score obtained from a multi-Med-LLM debate and from visual-tool confidence. The method is evaluated on four public datasets (SLAKE, VQA-RAD, IU-Xray, MIMIC-CXR) with LLaVA-Med-1.5 and LLaVA-Med++ backbones, comparing against DPO, POVID, FiSAO, SIMA, STLLaVA-Med, and a self-rewarding baseline. The paper reports consistent gains over these baselines and includes ablations for the two data-curation strategies, the clinical-relevance weighting, local versus global noise, and single versus multiple Med-LLM scoring.","tokens_in":18401,"tokens_out":4182,"duration_ms":40601,"significance":"If the central claim held, MMedPO would offer a practical recipe for making preference data more clinically meaningful in medical VLMs, and the reported gains on external benchmarks would be a useful advance. The paper has strengths: it uses standard public benchmarks, compares with several preference-optimization baselines, provides ablations, reports compatibility with a second backbone, and releases code. However, the central mechanism for the lesion-region-neglect preference pairs is not implemented as described, the abstract's headline numbers do not match the body, and several evidence claims are weaker than stated. These issues are load-bearing for the paper's main contribution, so the current version cannot be accepted without substantial revision.","major_comments":[{"comment":"For the Dv preference pairs, the preferred and dispreferred responses are the same ground-truth y: the text says 'the original image xv paired with the ground truth y is considered preferred, while the image with localized noise xk paired with the same ground truth y is regarded as dispreferred,' and Algorithm 1 stores {xv, x∗_v, y, sv}. Substituting yw = yl = y into Eq. (3) gives a DPO loss that contrasts log πθ(y|x) with log πθ(y|x∗) for the same response, i.e., a likelihood ratio between clean and locally noised inputs, not a preference between a correct response and a response that neglects lesion regions. The manuscript never constructs the claimed 'responses that overlook critical regions.' Since Table 8 attributes most of the report-generation gain to Stage 2, the explanation for the headline improvement is not supported by the construction actually specified. The authors should either generate yl as the target model's output on the noised image (a genuine neglect response) and use it in Eq. (3), or explicitly reframe Stage 2 as a robustness regularizer and provide corresponding evidence.","section":"§3.1, Eq. (2)–(3), Algorithm 1"},{"comment":"The abstract's claim of 'averaging 14.2% and 51.7% across the Med-VQA and report generation tasks' is not consistent with the body. Section 4.2 reports average improvements over the best baseline of 15.8% and 10.3% for open-ended and closed-ended VQA questions, and 61.9% and 26.0% for IU-Xray and MIMIC-CXR report generation. The 14.2% figure appears in the text as the average improvement over the SFT baseline across all four datasets, not as a Med-VQA average. These numbers must be reconciled in a single consistent presentation.","section":"Abstract and §4.2"},{"comment":"The paper repeatedly characterizes BLEU, ROUGE-L, and METEOR as measures of 'factual accuracy' for report generation. These are lexical-overlap metrics and do not measure factual correctness or clinical validity; a report can score high while containing clinically wrong statements, and vice versa. The abstract and conclusion accordingly overstate what the experiments demonstrate about factuality. The authors should either use clinically meaningful factuality metrics (e.g., radiology-specific factual error rates or expert evaluation) or restrict their claims to what the metrics support.","section":"§4.1 and Conclusion"},{"comment":"No error bars, standard deviations, or significance tests are reported for any of the experimental results. Several differences in Table 1 are small (for example, MIMIC-CXR ROUGE-L gains of about 1 point over DPO or FiSAO), so without repeated runs or statistical testing it is unclear which improvements are reliable. At minimum, the authors should report multiple seeds with variance and, where possible, significance tests for the main comparisons.","section":"Tables 1–4, 7–9"},{"comment":"The claim that MMedPO improves the 'clinical relevance' of responses (Figure 6) is evaluated by Med-LLM judges of the same type used in the multi-agent collaboration that produces the training weights. This supporting evidence is not independent of the training procedure, so it cannot be used as confirmatory evidence for the clinical-relevance improvement. The authors should either use held-out human expert evaluation or a distinct evaluator model not involved in training, and should report agreement with human judgments if Med-LLM scores are used.","section":"§4.4.1 and §3.2.1"}],"minor_comments":[{"comment":"Eq. (3) uses πo to denote the reference policy, whereas Eq. (1) uses πref; please unify the notation.","section":"§3.3"},{"comment":"There are typos and grammar issues: 'Quatify' instead of 'Quantify' in Algorithm 1, 'importance' instead of 'important' in Section 3.2.2, and the sentence 'These scores, serve as weights' has an extraneous comma.","section":"§3.2.2"},{"comment":"The caption says the table covers SLAKE, VQA-RAD, and IU-Xray, but the table also reports MIMIC-CXR; please include all four datasets in the caption.","section":"Table 1 caption"},{"comment":"The localization accuracy of the MedKLIP heatmaps h = T(xv) is never measured on the four datasets. Since the localized noise is applied exactly where the heatmap is nonzero, reporting a quantitative validation of the heatmaps (e.g., overlap with ground-truth lesion annotations, or a manual audit) would strengthen the interpretation of the Stage 2 experiments.","section":"§3.1, Eq. (2)"},{"comment":"The hyperparameters α, β, μ, σ², ξ_k, k, the number of debate rounds r, and the choice of Med-LLM judges are fixed values without sensitivity analysis. The claim that all hyperparameters are kept consistent does not address whether results depend strongly on these choices.","section":"Appendix B"},{"comment":"The figures and tables report an 'average score on each dataset' without defining how the average is computed across the different metrics for each dataset; please specify the aggregation formula.","section":"Figure 3 and Table 2"},{"comment":"The sentence 'As illustrated in Table 4' appears to refer to the compatibility analysis, but the compatibility results are shown in Figure 4; please correct the cross-reference.","section":"§4.3.4"}],"recommendation":"major_revision","confidential_remarks":"The central issue is fixable in principle: the authors need to construct genuine lesion-neglect dispreferred responses (e.g., responses generated on locally noised images) and then re-run the experiments with that corrected construction. If the reported gains persist under the corrected construction, the paper would be substantially stronger; if not, the Stage 2 gains should be reinterpreted as coming from a different mechanism. Given the abstract/body number mismatch and the absence of uncertainty quantification, the current version is not ready for acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take on MMedPO (arXiv:2412.06141). The clinical-relevance weighting of preference samples is a reasonable idea and the reported gains are consistently positive, but there is a load-bearing mismatch between the mechanism the paper claims and the data it actually constructs. Section 3.1 and Algorithm 1 store the same ground-truth text y as both y_w and y_l for the lesion-noising pairs (Dv). Plug into Eq. 3 and the DPO loss becomes a likelihood ratio for the same response on clean vs. noisy images—a robustness regularizer, not a preference between a correct response and one that neglects the lesion. Since Table 8 says that stage contributes most of the report-generation gain, the headline explanation is not supported by the construction. This is not a minor typo; it changes what the method is.\n\nWhat is genuinely new: integrating multi-agent Med-LLM scoring into weighted DPO, and using MedKLIP heatmaps to localize noise, is a sensible combination not present in DPO, POVID, FiSAO, SIMA, or STLLaVA-Med. The experiments cover four public benchmarks, the code is released, and the improvements over strong baselines are consistent. The ablation and compatibility checks are useful.\n\nThe soft spots beyond the main flaw: abstract numbers (14.2%, 51.7%) do not match the body (15.8%/10.3% and 61.9%/26.0%); there are no error bars or significance tests anywhere; report-generation 'factual accuracy' is measured with BLEU/ROUGE/METEOR, which are lexical metrics, not factuality metrics; the clinical-relevance evaluation in Figure 6 uses Med-LLMs of the same type that generated the training weights, so it is not independent; MDPO is cited but not compared; and the lesion masks from MedKLIP are never validated on these datasets.\n\nThe main fix is straightforward: either generate the target model's actual response on the noised image and use that as y_l, giving a genuine neglect response, or reframe the stage as input-perturbation robustness and drop the 'neglect' language. As written, the paper is internally contradictory, so I would not cite the claimed mechanism, but the empirical result is interesting enough that it deserves a serious referee. If I were the editor, I would send it out with a request to address the Dv construction and the statistical reporting.\n\nThis is a useful paper for a reading group—it is a clean example of a preference-data construction that looks right but is not.","headline":"The weighting idea is worth a look, but the lesion-noising stage as written compares the same ground-truth text on clean vs noisy images, so the paper's main mechanism is not what it claims.","tokens_in":18947,"tokens_out":3841,"would_cite":false,"duration_ms":41367,"reading_group":"yes","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"MMedPO claims clinical-relevance weighting of preference data improves medical VQA accuracy by 14.2% and report generation by 51.7% over prior methods.","keywords":["medical vision-language models","preference optimization","clinical relevance scoring","hallucination reduction","radiology report generation","medical visual question answering","weighted DPO","lesion localization"],"falsifier":"Measure the Dice overlap between the visual tool's 'disease' heatmaps and expert lesion segmentations on the four datasets, then compare MMedPO's margin over the global-noise ablation across images grouped by that overlap; if the margin disappears on low-overlap images, the localized-lesion mechanism is not doing the work attributed to it.","tokens_in":17846,"feed_emoji":"🩻","tokens_out":10785,"duration_ms":89565,"temperature":0.7,"pith_summary":"The paper claims that medical vision-language models hallucinate in part because preference-optimization training ignores how clinically meaningful each preference sample is, and it proposes MMedPO to fix that. MMedPO builds dispreferred samples by injecting plausible hallucinations into text responses and by adding noise specifically to lesion regions detected by a visual tool; it then scores each sample's clinical relevance using a panel of medical LLMs plus visual-tool confidence and feeds those scores into a weighted DPO loss. On VQA-RAD, SLAKE, MIMIC-CXR, and IU-Xray, the paper reports average gains of 14.2% on medical VQA and 51.7% on report generation over existing preference-optimization methods. If the claim holds, selecting which preference samples matter, and weighting them by clinical relevance, is a practical lever for reducing image-text misalignment and hallucinations in medical AI.","feed_headline":"Clinical-aware weighting lifts medical-VQA accuracy 14.2%","feed_subtitle":"And report generation by 51.7%: scoring each preference sample's clinical relevance beats DPO and its variants.","key_machinery":"The load-bearing object is a weighted direct preference optimization (DPO) loss, Eq. (3), in which each preference tuple $(x, x^*, y_w, y_l, s')$ contributes proportionally to its normalized clinical-relevance weight $s'$:\n$$\\mathcal{L}_{\\mathrm{MMedPO}} = -\\mathbb{E}_{(x,x^*,y_w,y_l,s')\\sim\\mathcal{D}_o}\\left[s' \\log\\$\\sigma$\\left(\\$\\alpha$ \\log\\frac{\\pi_\\$\\theta$(y_w|x)}{\\pi_o(y_w|x)} - \\$\\alpha$ \\log\\frac{\\pi_\\$\\theta$(y_l|x^*)}{\\pi_o(y_l|x^*)}\\right)\\right].$$\nThe weight $s'$ comes from one of two sources: a multi-agent Med-LLM consensus score for hallucinated-text dispreferred responses, or the visual tool's lesion-detection confidence for image-noise pairs, and is z-score normalized and clipped to $[\\alpha,\\beta]=[0.75,1.25]$ before use. The noised image in the second preference source is built by Eq. (2):\n$$x_v^* = \\sqrt{\\bar{\\xi}_k}(x_v\\odot h) + \\sqrt{1-\\bar{\\xi}_k}(\\epsilon\\odot h) + (x_v\\odot(1-h)),$$\nwhere $h$ is the lesion-region heatmap from the visual tool, so only the detected disease area is perturbed and the rest of the image is left intact.","core_discovery":"On the paper's own terms, the central discovery is that preference optimization for Med-LVLMs becomes more effective when the training signal is clinical rather than generic. The authors construct dispreference in two complementary ways: hallucinated medical answers selected or generated through GPT-4o, and lesion-region neglect produced by locally noising the heatmap-detected disease area. They then weigh each preference pair by a quantified clinical relevance score—multi-agent Med-LLM consensus for text responses, visual-tool confidence for lesion localization—and run a normalized weighted DPO. The reported result is that this pipeline outperforms DPO, POVID, FiSAO, SIMA, STLLaVA-Med, and self-rewarding baselines on all four datasets, with the largest margins on open-ended and report-generation tasks.","pith_inferences":["The z-score normalization with fixed clipping bounds $[0.75,1.25]$ is a free hyperparameter; testing wider or dataset-dependent bounds would reveal how sensitive the reported gains are to the weighting schedule.","The clinical scorer sees text only for hallucinated-text samples; integrating the lesion heatmaps into that score could catch cases where a plausible answer contradicts the image, which the current pipeline may miss.","If the lesion-localization mechanism is what carries the report-generation gain, the same recipe should transfer to CT, ultrasound, or pathology slides whenever a heatmap tool exists; a rerun on such data would directly test that transfer.","The 51.7% report-generation improvement is measured with lexical metrics (BLEU/ROUGE/METEOR); a structured clinical-evaluation study, such as checking whether generated reports match expert-labeled findings, would show whether the gain translates to clinically meaningful text."],"forward_implications":["Weighting preference samples by clinical relevance is a model-agnostic upgrade: MMedPO improves both LLaVA-Med-1.5 and LLaVA-Med++ without changing the backbone.","Localized lesion-noising is the stronger ingredient on report generation, so datasets dominated by free-form image-to-text pairs benefit most from visual-tool-guided dispreference.","Clinical-relevance weights provide a clear gain over unweighted DPO, with an average 2.3% improvement on VQA and 18.5% on report generation in the ablation study.","Multi-agent consensus scoring beats single Med-LLM scoring by an average of 3.6%, indicating that score reliability directly affects alignment quality.","Combining hallucinated-text pairs and lesion-region pairs outperforms either preference source alone, so the two mechanisms are complementary."],"supporting_citations":[{"why":"Defines the DPO objective that MMedPO reweights; the optimization foundation for all preference baselines.","marker":"Rafailov et al. 2023"},{"why":"MedKLIP supplies the disease heatmaps and confidence scores used for localized lesion-noising and sample weighting.","marker":"Wu et al. 2023b"},{"why":"LLaVA-Med-1.5 is the main base Med-LVLM that MMedPO fine-tunes.","marker":"Li et al. 2023a"},{"why":"Provides VQA-RAD, one of the two Med-VQA evaluation datasets.","marker":"Lau et al. 2018"},{"why":"Provides SLAKE, the other Med-VQA evaluation dataset.","marker":"Liu et al. 2021"},{"why":"Provides MIMIC-CXR, one of the two report-generation evaluation datasets.","marker":"Johnson et al. 2020"},{"why":"Provides IU-Xray, the other report-generation evaluation dataset.","marker":"Demner-Fushman et al. 2016"},{"why":"GPT-4o selects and generates hallucinated dispreferred responses in the preference data curation.","marker":"OpenAI 2023"},{"why":"Med42-7B/70B are among the Med-LLMs in the multi-agent clinical relevance scoring.","marker":"Christophe et al. 2024"},{"why":"BioMistral-7B is another Med-LLM used in the multi-agent clinical relevance scoring.","marker":"Labrak et al. 2024"}],"fun_headline_variants":["MMedPO: clinical relevance weighting lifts Med-VQA 14.2%","Clinical-aware preference optimization boosts report generation 51.7%","Weighted clinical preference data improves medical VQA and reports","MMedPO beats DPO with clinically relevant preference pairs","Clinical relevance scoring enhances multimodal medical alignment"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The training signal from lesion-localized noise is only as good as the visual tool's disease heatmaps: if the tool misses or misplaces the lesion, the noised image teaches the model to ignore the true diagnostic region, and the confidence-score weights may reward that mislocalization.","fun_headline_variants_meta":{"raw":{"variants":["MMedPO: clinical relevance weighting lifts Med-VQA 14.2%","Clinical-aware preference optimization boosts report generation 51.7%","Weighted clinical preference data improves medical VQA and reports","MMedPO beats DPO with clinically relevant preference pairs","Clinical relevance scoring enhances multimodal medical alignment"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000167,"raw_usage":{"total_tokens":1277,"prompt_tokens":982,"completion_tokens":295,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":598,"completion_tokens_details":{"reasoning_tokens":212}},"tokens_in":598,"tokens_out":295,"duration_ms":3623,"temperature":1.0,"reasoning_tokens":212,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T19:59:09.092251+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the Dice overlap between the visual tool's 'disease' heatmaps and expert lesion segmentations on the four datasets, then compare MMedPO's margin over the global-noise ablation across images grouped by that overlap; if the margin disappears on low-overlap images, the localized-lesion mechanism is not doing the work attributed to it.","supporting_citations":[{"cited_title":"Slake: A semantically-labeled knowledge-enhanced dataset for medical visual question answering","cited_arxiv_id":null,"evidence_quote":"Provides SLAKE, the other Med-VQA evaluation dataset."}],"review_version":1}