Pith. sign in

REVIEW 4 major objections 6 minor 3 cited by

Improving Medical Reasoning with Curriculum-Aware Reinforcement Learning

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A two-stage reinforcement-learning curriculum — close-ended questions first, open-ended second — lifts medical VQA accuracy to 62.3% in-domain and 41.7% out-of-domain, beating prior medical VLMs on 8 of 11 sub-tasks.

desk verdict Useful curriculum-GRPO ablation for medical VQA, but the open-ended results are partly self-scoring because the training reward is also the evaluation metric. read the letter →

arxiv 2505.19213 v1 pith:ML6K4OUM submitted 2025-05-25 cs.AI

Shaohao Rui , Kaitao Chen , Weijie Ma , Xiaosong Wang This is my paper
classification cs.AI
keywords medicalVQAreinforcementlearningGRPOcurriculumopen-endedreasoningmultimodalLLMrewarddesignout-of-domaingeneralization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

MedCCO claims that teaching a medical vision-language model to reason is best done in two reinforcement-learning stages ordered by difficulty. Stage one applies Group Relative Policy Optimization (GRPO) — a reinforcement-learning algorithm that ranks a group of sampled answers rather than learning a value function — to close-ended medical VQA with a strict right-or-wrong accuracy reward. Stage two switches to open-ended VQA and rewards answers with a blended score of word overlap (BLEU-1, ROUGE-1) and semantic similarity (BERTScore). Across eight benchmarks, MedCCO-7B averages 62.3% in-domain and 41.7% out-of-domain accuracy and beats the strongest baseline on 8 of 11 sub-tasks, all without hand-written reasoning chains. The paper explains the gain by a conflict between discrete and continuous reward signals: training both task types at once destabilizes learning, while ordering the tasks by difficulty avoids the conflict.

What carries the argument

The load-bearing object is the two-stage curriculum GRPO schedule built on Group Relative Policy Optimization, a value-free reinforcement-learning algorithm that normalizes each answer's reward against a group of sampled answers and optimizes a clipped policy objective against a reference policy. Stage one uses the binary close-ended reward $R_{\text{close}}$ (1 if the predicted answer equals the ground truth, else 0). Stage two uses the open-ended composite reward $$R_{\text{open}}=\tfrac{1}{2}\big[\$\lambda$\,(\text{BLEU-1}+\text{ROUGE-1})+(1-\$\lambda$)\,\text{BERTScore}\big]$$ with $\lambda=0.7$, plus a format reward enforcing the <think>...</think> and <answer>...</answer> structure, and the total reward blends content and format with weight $\gamma=0.8$. Before stage two, a VQA-Consistency Auditor — a 72-billion-parameter vision-language model — rewrites open-ended question-answer pairs so that the granularity of each answer matches what the question asks. The paper's contrast case is joint GRPO, which balances task gradients by batch-size ratio, and the argued mechanism is that sequencing the tasks avoids a gradient conflict between discrete and continuous reward signals.

What would settle it

Score a sample of open-ended test responses from MedCCO and its strongest baselines with Eq. (4), then have independent clinicians rank the same answers for clinical correctness: if the clinician ordering does not track the composite ordering, the open-ended claim collapses. A cheaper check is to train the identical backbone on open-ended data only, or with the curriculum reversed (open-ended first, then close-ended); if either schedule matches or beats MedCCO's reported numbers, the close-to-open ordering is not the cause of the gains.

Watch

Extended reading notes

Core claim

The central claim is procedural and comparative: a curriculum-driven GRPO schedule that trains first on close-ended medical VQA and then on open-ended VQA outperforms vanilla GRPO, a joint GRPO scheme with gradient re-weighting, supervised fine-tuning, and previously published medical vision-language models, on both in-domain and out-of-domain benchmarks. MedCCO-7B averages 62.3% accuracy on the three in-domain test sets (VQA-RAD, SLAKE, PathVQA) and 41.7% on the three out-of-domain benchmarks (Quilt-VQA, PMC-VQA, MedXpertQA), beating the strongest baseline on 8 of 11 sub-tasks and scoring 59.3% on the MMMU Health and Medicine track. The authors attribute the ordering advantage to the differing reward geometry of the two task types: binary rewards give sharp, high-variance advantages while continuous composite rewards give smoother ones, and joint training lets the two clash. They further report that refining open-ended question-answer pairs so that question granularity matches the reference answer stabilizes RL training and adds several points, and that the recipe transfers across imaging modalities, reaching 92.7% accuracy on MRI and 72.5% on CT after training on X-ray images only.

Load-bearing premise

The load-bearing premise is that the composite score in Eq. (4) — a blend of BLEU-1, ROUGE-1, and BERTScore comparing the model's free-text answer with a short reference answer — measures whether a medical answer is actually right, and that training and evaluating open-ended ability on this same score is fair; if fluent wrong answers can score high, or correct answers phrased differently can score low, the reported open-ended gains do not demonstrate better medical reasoning.

Editorial extensions

If this is right

  • If the central claim holds, sequential RL — closed-form grounding first, generative free-text refinement second — becomes a reusable training recipe for medical multimodal models that need both accurate discrimination and explainable answers, with no chain-of-thought annotations required.
  • The reward-conflict mechanism, if right, predicts that any mixed discrete/continuous reward schedule benefits from curriculum ordering or an explicit conflict-mitigation technique, a prediction that can be carried to other multi-task RL setups.
  • GRPO-based fine-tuning is presented as generalizing out-of-domain better than supervised fine-tuning, because RL explores reasoning pathways instead of memorizing input-output mappings.
  • Aligning question and answer granularity in the training data is reported to be a substantial lever for open-ended RL, worth up to 3.9% in-domain and 3.1% out-of-domain in the ablations.
  • The cross-modal results (92.7% MRI and 72.5% CT accuracy after training on X-ray images only) indicate that curriculum-driven RL builds transferable, modality-agnostic medical knowledge rather than surface visual mappings.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: the close-to-open curriculum is a candidate general recipe for any domain that mixes closed-form evaluation with free-form generation, such as legal or scientific question answering, since the paper's reward-conflict argument is not medical in character.
  • Editorial extension: the stated gradient-conflict mechanism can be tested directly by tracking the cosine similarity between close-ended and open-ended task gradients across training; the paper infers the conflict from final accuracy differences rather than observing it.
  • Editorial extension: the missing control is the reversed curriculum (open-ended first, then close-ended); if reversal matched the proposed order's accuracy, the difficulty-ordering explanation would need revision.
  • Editorial extension: the paper itself ends by calling automated evaluation of reasoning quality an open problem, so a clinician-graded study of MedCCO's free-text answers is the immediate next test of whether these gains mean better clinical reasoning.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces MedCCO, a GRPO-based curriculum reinforcement fine-tuning framework for medical VQA. MedCCO first trains a Qwen2.5-VL backbone on close-ended VQA tasks with an exact-match accuracy reward, then adapts it to open-ended VQA tasks using a hybrid lexical/semantic reward (Eq. 4). The authors compare MedCCO against general and medical VLMs as well as SFT and vanilla GRPO baselines on eight benchmarks, reporting state-of-the-art or competitive results on in-domain and out-of-domain tests (Tables 2 and 3), and an ablation showing curriculum GRPO outperforms joint GRPO (Table 4). The central claim is that this curriculum schedule improves both discriminative and generative medical reasoning without hand-crafted reasoning chains.

Significance. If the open-ended results were validated with an independent metric, the paper would make a useful contribution: it demonstrates a simple but effective sequential RL strategy for medical VLMs and provides a first internal comparison between joint and curriculum GRPO (Table 4) in a medical setting. The authors release their code on GitHub, which supports reproducibility. However, because the open-ended evaluation metric is the very reward used during training, the reported gains on open-ended tasks do not yet establish improved clinical reasoning; the significance hinges on whether the composite score tracks clinically meaningful correctness.

major comments (4)
  1. [§3.2, Eq. (4); Table 2] The open-ended evaluation metric in Table 2 is the same weighted composite (BLEU-1, ROUGE-1, BERTScore) that defines R_open in Eq. (4), and the paper states in Section 5 that automatically evaluating reasoning quality remains an open problem. Consequently, the large open-ended improvements (e.g., SLAKE open-ended 65.7 vs. 40.2 for the GRPO baseline in Table 2) partly reflect optimizing the test metric itself, not necessarily better medical reasoning. The authors should either add an independent validation (e.g., expert clinician scoring, a held-out answer-equivalence judge that was not used as a reward, or a separately-specified reasoning metric) or soften the claim to 'improved scores on the training-aligned metric'.
  2. [§4, Tables 2 and 4] All reported numbers appear to come from a single training run; no error bars or multiple seeds are given. Given the inherent variance of GRPO and the small margins in some key comparisons (e.g., Table 4: 7B J.GRPO vs. C.GRPO in-domain 77.5 vs. 79.5; VQA-RAD 72.1 vs. 76.3), the claim that MedCCO 'consistently outperforms' the baselines is not statistically supported. Please report mean and standard deviation over at least three seeds for the main tables.
  3. [§3.3; Table 4] The curriculum-versus-joint comparison is the central ablation for the paper's headline claim, but the description does not establish that all training settings except task ordering are held constant. Please specify the exact number of training steps (or epochs) per stage for curriculum, whether the joint GRPO baseline uses the same data mixture, the same learning rate schedule, and the same GRPO hyperparameters (G, β, λ, γ), and whether the vanilla GRPO baseline in Table 2 is identical to the joint baseline in Table 4. Without this, the observed differences might be due to training duration or data composition rather than the curriculum itself.
  4. [§3.2, Eq. (4); §4] The choice of BLEU-1, ROUGE-1, and BERTScore as the open-ended reward is questionable for medical answers with very short reference strings (mean length ≈5 tokens). Clinically correct answers that use different phrasing can receive low scores, while fluent but incorrect statements can match the reference lexical/semantic content. The authors should provide evidence that the composite score correlates with clinical correctness (e.g., error analysis or a small human-rated sample), or adopt a more robust answer-equivalence mechanism.
minor comments (6)
  1. [Table 4] The symbol '%' in the row '% J.GRPO' appears to be a typo for '✗' (meaning no VQA refinement); please fix.
  2. [Table 2 caption] For the open-ended columns, please state the exact weights (λ=0.7) and the BERTScore settings (e.g., model, whether rescaled) used for evaluation.
  3. [Section 4.1] Please define what the 'R.c.' and 'U.c.' columns mean for PMC-VQA in Table 2 (they appear to denote reasoning/understanding with close-ended options).
  4. [Section 3.2] The total reward is written as 'γR + (1−γ)R_format' without an explicit equation number; please label it (e.g., Eq. 5) for clarity.
  5. [Section 4.2] In Table 4, please clarify whether the in-domain average includes both close-ended and open-ended scores or only close-ended accuracy; the numbers in the first three columns match the close-ended columns of Table 2, suggesting the average might be misleading.
  6. [Abstract] The claim of being 'the first' multimodal medical reasoning model for both close- and open-ended VQA should be supported by a more thorough related-work comparison or softened, given rapidly evolving work in this area.

Circularity Check

1 steps flagged · score 6.0 of 10

Open-ended results are evaluated with the same combined lexical/semantic metric used as the GRPO training reward, so the reported open-ended gains partly reduce to optimizing the test metric.

  1. self definitional [Section 3.2 (Eq. 4), Section 3.2 Total Reward, Table 2 caption]
    "For open-ended responses, we design a hybrid reward function ... Ropen(o, g) = 1/2 λ·(BLEU1(o,g)+ROUGE1(o,g)) + (1−λ)·BERTScore(o,g), (4) ... For each sample, the total reward is computed as γR + (1−γ)Rformat, where R denotes Rclose for close-ended and Ropen for open-ended questions. Table 2 caption: o.: open-ended metrics (combined BLEU1, ROUGE1 and BERTScore.)."

    The open-ended evaluation columns are exactly the same function the GRPO objective maximizes: Eq. (4) is the reward R_open, and the total training reward is γ·R_open + (1−γ)·R_format. So higher Table 2 open-ended metrics are partly guaranteed by reward optimization rather than by clinically better reasoning. The paper reports no clinician scoring or independent held-out metric, and its own conclusion states that automatically evaluating reasoning quality remains an open problem. Thus the open-ended part of the central claim reduces, by construction, to optimizing the evaluation metric; the close-ended exact-match results are not circular.

full rationale

The central claim has two separable halves. The close-ended half is not circular: Eq. (3) uses exact-match correctness as the reward, and Table 2's close-ended columns report exact accuracy, so those gains are measured on an independent criterion. The curriculum-versus-joint comparison is also an internal empirical ablation, not a circularity. However, the open-ended half is partially circular: the reward designed in Eq. (4), a weighted combination of BLEU-1, ROUGE-1 and BERTScore, is reused verbatim as the evaluation metric in every open-ended column of Table 2. Because RL directly maximizes that same composite (with an additional format term), the reported open-ended improvements partly reflect better optimization of the evaluation function itself. The paper does not supply a separate validation of this metric, and its own concluding paragraph concedes that automatically evaluating reasoning quality remains an open problem. This makes the open-ended claim score a 6 under the rubric: one class of predictions reduces by construction to its own optimization target, while the close-ended evidence remains independent.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

No new physical or architectural entities are introduced. The VQA-Consistency Auditor is an existing model (Qwen2.5-VL-72B) used as a data-processing tool, not a new postulated entity, so the invented-entities ledger is empty.

free parameters (3)
  • lambda = 0.7
    Weight in Eq (4) balancing lexical metrics against BERTScore for the open-ended reward; chosen by hand without sensitivity analysis.
  • gamma = 0.8
    Weight combining the correctness reward with the format reward in the total reward; fixed in Implementation Details without sensitivity analysis.
  • beta (KL penalty coefficient) = 0.01
    Regularization strength in Eq (2) for the GRPO objective; chosen by hand and reported without ablations.
assumptions (4)
  • domain assumption GRPO with rule-based rewards trains VLMs effectively for medical VQA
    The method transplants GRPO from general-domain VLM/LLM reasoning to medical imaging and assumes the same convergence and generalization behavior holds.
  • domain assumption The composite BLEU-1/ROUGE-1/BERTScore reward is a valid measure of open-ended medical answer correctness
    Eq (4) defines the open-ended reward, and Table 2 uses the same composite as the open-ended evaluation metric.
  • domain assumption Qwen2.5-VL-72B auditor rewrites VQA pairs without changing clinical meaning
    Section 3.4 uses the auditor to refine training data; if the auditor alters answer semantics, rewards are computed against modified references.
  • standard math Advantage normalization in Eq (1) and KL regularization in Eq (2) are appropriate for this setting
    Standard GRPO machinery is applied without modification, relying on standard policy-gradient theory.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Improving Medical Reasoning with Curriculum-Aware Reinforcement Learning." pith.science (2026). https://pith.science/paper/ML6K4OUM

@misc{pith2026250519213,
  author       = {Pith},
  title        = {Pith review of: Improving Medical Reasoning with Curriculum-Aware Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ML6K4OUM}},
  note         = {Machine review of arXiv:2505.19213}
}
read the original abstract

Recent advances in reinforcement learning with verifiable, rule-based rewards have greatly enhanced the reasoning capabilities and out-of-distribution generalization of VLMs/LLMs, obviating the need for manually crafted reasoning chains. Despite these promising developments in the general domain, their translation to medical imaging remains limited. Current medical reinforcement fine-tuning (RFT) methods predominantly focus on close-ended VQA, thereby restricting the model's ability to engage in world knowledge retrieval and flexible task adaptation. More critically, these methods fall short of addressing the critical clinical demand for open-ended, reasoning-intensive decision-making. To bridge this gap, we introduce \textbf{MedCCO}, the first multimodal reinforcement learning framework tailored for medical VQA that unifies close-ended and open-ended data within a curriculum-driven RFT paradigm. Specifically, MedCCO is initially fine-tuned on a diverse set of close-ended medical VQA tasks to establish domain-grounded reasoning capabilities, and is then progressively adapted to open-ended tasks to foster deeper knowledge enhancement and clinical interpretability. We validate MedCCO across eight challenging medical VQA benchmarks, spanning both close-ended and open-ended settings. Experimental results show that MedCCO consistently enhances performance and generalization, achieving a 11.4\% accuracy gain across three in-domain tasks, and a 5.7\% improvement on five out-of-domain benchmarks. These findings highlight the promise of curriculum-guided RL in advancing robust, clinically-relevant reasoning in medical multimodal language models.

Figures

Figures reproduced from arXiv: 2505.19213 by the authors.

Figure 1
Figure 1. Overview of our MedCCO. MedCCO is first fine-tuned with GRPO on close-ended medical [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Cross-modal performance on SLAKE [16], with each model trained on a single modality and evaluated across all modalities for in- and cross-modal comparison. scan based on their spatial location and morphological features. For the close-ended example (c), MedCCO handles a clinically grounded diagnostic task by selecting surgical excision as the next management step for a shoulder mass, justifying its decision with mul… view at source ↗
Figure 3
Figure 3. Training curves and overall performance with and without VQA refinement. Incorporating [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Qualitative results of open-ended and close-ended VQA reasoning. (a) and (b) show [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Prompt for refining open-ended VQA consistency. [PITH_FULL_IMAGE:figures/full_fig_p015_5.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. A global log for medical AI

    cs.AI 2025-10 conditional novelty 6.0 of 10

    MedLog defines a nine-field, syslog-style event log for clinical AI, intended to support real-world surveillance and auditing; the four-deployment validation claimed in the abstract is absent from the body.

  2. AdaThink-Med: Optimizing Inference-Time Compute for Medical Reasoning via Uncertainty Quantification

    cs.CL 2025-09 conditional novelty 6.0 of 10

    An entropy-guided reward that shortens answers to easy, confident questions and extends reasoning on hard ones reduces medical QA tokens by 4.7x to 6.4x with minimal accuracy loss.

  3. CX-Mind: A Pioneering Multimodal Large Language Model for Interleaved Reasoning in Chest X-ray via Curriculum-Guided Reinforcement Learning

    cs.LG 2025-07 conditional novelty 5.0 of 10

    CX-Mind combines curriculum reinforcement learning and rule-based process rewards to train a chest X-ray vision-language model that produces interleaved think-answer reasoning and reports state-of-the-art results acro...

Reference graph

Works this paper leans on

45 extracted references · 1 canonical work pages · cited by 3 Pith papers

  1. [1]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report.arXiv preprint arXiv:2502.13923, 2025

    arXiv 2025

  2. [2]

    Curriculum learning

    Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. Curriculum learning. InProceedings of the 26th annual international conference on machine learning, pages 41–48, 2009

    2009

  3. [3]

    Huatuogpt-vision, towards injecting medical visual knowledge into multimodal llms at scale.arXiv preprint arXiv:2406.19280, 2024

    Junying Chen, Chi Gui, Ruyi Ouyang, Anningzhe Gao, Shunian Chen, Guiming Hardy Chen, Xidong Wang, Ruifei Zhang, Zhenyang Cai, Ke Ji, et al. Huatuogpt-vision, towards injecting medical visual knowledge into multimodal llms at scale.arXiv preprint arXiv:2406.19280, 2024

    arXiv 2024

  4. [4]

    Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling.arXiv preprint arXiv:2412.05271, 2024

    Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shen- glong Ye, Hao Tian, Zhaoyang Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling.arXiv preprint arXiv:2412.05271, 2024

    arXiv 2024

  5. [5]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025

    DeepSeek-AI. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025

    2025

  6. [6]

    Virgo: A preliminary exploration on reproducing o1-like mllm.arXiv preprint arXiv:2501.01904, 2025

    Yifan Du, Zikang Liu, Yifan Li, Wayne Xin Zhao, Yuqi Huo, Bingning Wang, Weipeng Chen, Zheng Liu, Zhongyuan Wang, and Ji-Rong Wen. Virgo: A preliminary exploration on reproducing o1-like mllm.arXiv preprint arXiv:2501.01904, 2025

    arXiv 2025

  7. [7]

    Pathvqa: 30000+ questions for medical visual question answering.arXiv preprint arXiv:2003.10286, 2020

    Xuehai He, Yichen Zhang, Luntian Mou, Eric Xing, and Pengtao Xie. Pathvqa: 30000+ questions for medical visual question answering.arXiv preprint arXiv:2003.10286, 2020

    arXiv 2003

  8. [8]

    Omnimed- vqa: A new large-scale comprehensive evaluation benchmark for medical lvlm

    Yutao Hu, Tianbin Li, Quanfeng Lu, Wenqi Shao, Junjun He, Yu Qiao, and Ping Luo. Omnimed- vqa: A new large-scale comprehensive evaluation benchmark for medical lvlm. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22170–22183, 2024

    2024

Show all 45 references
  1. [9]

    Vision-r1: Incentivizing reasoning capability in multimodal large language models.arXiv preprint arXiv:2503.06749, 2025

    Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao, Zheyu Ye, Fei Zhao, Zhe Xu, Yao Hu, and Shaohui Lin. Vision-r1: Incentivizing reasoning capability in multimodal large language models.arXiv preprint arXiv:2503.06749, 2025

    2025 arXiv

  2. [10]

    Gonzalez, Hao Zhang, and Ion Stoica

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large lan- guage model serving with pagedattention. InProceedings of the ACM SIGOPS 29th Symposium on Operating System...

    2023

  3. [11]

    Med-r1: Reinforce- ment learning for generalizable medical reasoning in vision-language models.arXiv preprint arXiv:2503.13939, 2025

    Yuxiang Lai, Jike Zhong, Ming Li, Shitian Zhao, and Xiaofeng Yang. Med-r1: Reinforce- ment learning for generalizable medical reasoning in vision-language models.arXiv preprint arXiv:2503.13939, 2025

    2025

  4. [12]

    A dataset of clinically generated visual questions and answers about radiology images.Scientific data, 5(1):1–10, 2018

    Jason J Lau, Soumya Gayen, Asma Ben Abacha, and Dina Demner-Fushman. A dataset of clinically generated visual questions and answers about radiology images.Scientific data, 5(1):1–10, 2018

    2018

  5. [13]

    Llava-med: Training a large language-and-vision assistant for biomedicine in one day.Advances in Neural Information Processing Systems, 36:28541–28564, 2023

    Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Naumann, Hoifung Poon, and Jianfeng Gao. Llava-med: Training a large language-and-vision assistant for biomedicine in one day.Advances in Neural Information Processing Systems, 36:28541–28564, 2023

    2023

  6. [14]

    Llava-med: Training a large language-and-vision assistant for biomedicine in one day.Advances in Neural Information Processing Systems, 36:28541–28564, 2023

    Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Naumann, Hoifung Poon, and Jianfeng Gao. Llava-med: Training a large language-and-vision assistant for biomedicine in one day.Advances in Neural Information Processing Systems, 36:28541–285...

    2023

  7. [15]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. InInternational conference on machine learning, pages 19730–19742. PMLR, 2023

    2023

  8. [16]

    Slake: A semantically- labeled knowledge-enhanced dataset for medical visual question answering

    Bo Liu, Li-Ming Zhan, Li Xu, Lin Ma, Yan Yang, and Xiao-Ming Wu. Slake: A semantically- labeled knowledge-enhanced dataset for medical visual question answering. In2021 IEEE 18th international symposium on biomedical imaging (ISBI), pages 1650–1654. IEEE, 2021

    2021

  9. [17]

    Llava-next: Improved reasoning, ocr, and world knowledge, January 2024

    Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llava-next: Improved reasoning, ocr, and world knowledge, January 2024

    2024

  10. [18]

    Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023

    2023

  11. [19]

    Seg- zero: Reasoning-chain guided segmentation via cognitive reinforcement.arXiv preprint arXiv:2503.06520, 2025

    Yuqi Liu, Bohao Peng, Zhisheng Zhong, Zihao Yue, Fanbin Lu, Bei Yu, and Jiaya Jia. Seg- zero: Reasoning-chain guided segmentation via cognitive reinforcement.arXiv preprint arXiv:2503.06520, 2025

    2025 arXiv

  12. [20]

    Med-flamingo: a multimodal medical few-shot learner

    Michael Moor, Qian Huang, Shirley Wu, Michihiro Yasunaga, Yash Dalmia, Jure Leskovec, Cyril Zakka, Eduardo Pontes Reis, and Pranav Rajpurkar. Med-flamingo: a multimodal medical few-shot learner. InMachine Learning for Health (ML4H), pages 353–367. PMLR, 2023

    2023

  13. [21]

    Medvlm-r1: Incentivizing medical reasoning capability of vision-language models (vlms) via reinforcement learning.arXiv preprint arXiv:2502.19634, 2025

    Jiazhen Pan, Che Liu, Junde Wu, Fenglin Liu, Jiayuan Zhu, Hongwei Bran Li, Chen Chen, Cheng Ouyang, and Daniel Rueckert. Medvlm-r1: Incentivizing medical reasoning capability of vision-language models (vlms) via reinforcement learning.arXiv preprint arXiv:2502.19634, 2025

    2025 arXiv

  14. [22]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. InInternational conference on machine learning, pag...

    2021

  15. [23]

    Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347, 2017

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  16. [24]

    Quilt-llava: Visual instruction tuning by extracting localized narratives from open- source histopathology videos

    Mehmet Saygin Seyfioglu, Wisdom O Ikezogwo, Fatemeh Ghezloo, Ranjay Krishna, and Linda Shapiro. Quilt-llava: Visual instruction tuning by extracting localized narratives from open- source histopathology videos. InProceedings of the IEEE/CVF Conference on Computer Vision and Pa...

    2024

  17. [25]

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024

    2024 arXiv

  18. [26]

    Vlm-r1: A stable and generalizable r1-style large vision-language model.arXiv preprint arXiv:2504.07615, 2025

    Haozhan Shen, Peng Liu, Jingcheng Li, Chunxin Fang, Yibo Ma, Jiajia Liao, Qiaoli Shen, Zilun Zhang, Kangjia Zhao, Qianqian Zhang, et al. Vlm-r1: A stable and generalizable r1-style large vision-language model.arXiv preprint arXiv:2504.07615, 2025

    2025 arXiv

  19. [27]

    Hybridflow: A flexible and efficient rlhf framework.arXiv preprint arXiv: 2409.19256, 2024

    Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework.arXiv preprint arXiv: 2409.19256, 2024

    2024 arXiv

  20. [28]

    Aligning large multimodal models with factually augmented rlhf.arXiv preprint arXiv:2309.14525, 2023

    Zhiqing Sun, Sheng Shen, Shengcao Cao, Haotian Liu, Chunyuan Li, Yikang Shen, Chuang Gan, Liang-Yan Gui, Yu-Xiong Wang, Yiming Yang, et al. Aligning large multimodal models with factually augmented rlhf.arXiv preprint arXiv:2309.14525, 2023

    2023 arXiv

  21. [29]

    Qwq-32b: Embracing the power of reinforcement learning, March 2025

    Qwen Team. Qwq-32b: Embracing the power of reinforcement learning, March 2025

    2025

  22. [30]

    Visualprm: An effective process reward model for multimodal reasoning.arXiv preprint arXiv:2503.10291, 2025

    Weiyun Wang, Zhangwei Gao, Lianjie Chen, Zhe Chen, Jinguo Zhu, Xiangyu Zhao, Yangzhou Liu, Yue Cao, Shenglong Ye, Xizhou Zhu, et al. Visualprm: An effective process reward model for multimodal reasoning.arXiv preprint arXiv:2503.10291, 2025

    2025 arXiv

  23. [31]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837, 2022. 11

    2022

  24. [32]

    Towards generalist foundation model for radiology by leveraging web-scale 2d&3d medical data.arXiv preprint arXiv:2308.02463, 2023

    Chaoyi Wu, Xiaoman Zhang, Ya Zhang, Yanfeng Wang, and Weidi Xie. Towards generalist foundation model for radiology by leveraging web-scale 2d&3d medical data.arXiv preprint arXiv:2308.02463, 2023

    2023 arXiv

  25. [33]

    Medreason: Eliciting factual medical reasoning steps in llms via knowledge graphs.arXiv preprint arXiv:2504.00993, 2025

    Juncheng Wu, Wenlong Deng, Xingxuan Li, Sheng Liu, Taomian Mi, Yifan Peng, Ziyang Xu, Yi Liu, Hyunjin Cho, Chang-In Choi, et al. Medreason: Eliciting factual medical reasoning steps in llms via knowledge graphs.arXiv preprint arXiv:2504.00993, 2025

    2025 arXiv

  26. [34]

    Yi: Open foundation models by 01

    Alex Young, Bei Chen, Chao Li, Chengen Huang, Ge Zhang, Guanwei Zhang, Guoyin Wang, Heng Li, Jiangcheng Zhu, Jianqun Chen, et al. Yi: Open foundation models by 01. ai.arXiv preprint arXiv:2403.04652, 2024

    2024 arXiv

  27. [35]

    Finemedlm-o1: Enhancing the medical reasoning ability of llm from supervised fine-tuning to test-time training.arXiv preprint arXiv:2501.09213, 2025

    Hongzhou Yu, Tianhao Cheng, Ying Cheng, and Rui Feng. Finemedlm-o1: Enhancing the medical reasoning ability of llm from supervised fine-tuning to test-time training.arXiv preprint arXiv:2501.09213, 2025

    2025 arXiv

  28. [36]

    Rlhf-v: Towards trustworthy mllms via behavior alignment from fine-grained correctional human feedback

    Tianyu Yu, Yuan Yao, Haoye Zhang, Taiwen He, Yifeng Han, Ganqu Cui, Jinyi Hu, Zhiyuan Liu, Hai-Tao Zheng, Maosong Sun, et al. Rlhf-v: Towards trustworthy mllms via behavior alignment from fine-grained correctional human feedback. InProceedings of the IEEE/CVF Conference on Com...

    2024

  29. [37]

    Rlaif-v: Aligning mllms through open-source ai feedback for super gpt-4v trustworthiness.arXiv preprint arXiv:2405.17220, 2024

    Tianyu Yu, Haoye Zhang, Yuan Yao, Yunkai Dang, Da Chen, Xiaoman Lu, Ganqu Cui, Taiwen He, Zhiyuan Liu, Tat-Seng Chua, et al. Rlaif-v: Aligning mllms through open-source ai feedback for super gpt-4v trustworthiness.arXiv preprint arXiv:2405.17220, 2024

    2024

  30. [38]

    Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi

    Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, et al. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. InProceedings of the IEEE/CVF Conference on Co...

    2024

  31. [39]

    R1-vl: Learning to reason with multimodal large language models via step-wise group relative policy optimization.arXiv preprint arXiv:2503.12937, 2025

    Jingyi Zhang, Jiaxing Huang, Huanjin Yao, Shunyu Liu, Xikun Zhang, Shijian Lu, and Dacheng Tao. R1-vl: Learning to reason with multimodal large language models via step-wise group relative policy optimization.arXiv preprint arXiv:2503.12937, 2025

    2025 arXiv

  32. [40]

    Pmc-vqa: Visual instruction tuning for medical visual question answering.arXiv preprint arXiv:2305.10415, 2023

    Xiaoman Zhang, Chaoyi Wu, Ziheng Zhao, Weixiong Lin, Ya Zhang, Yanfeng Wang, and Weidi Xie. Pmc-vqa: Visual instruction tuning for medical visual question answering.arXiv preprint arXiv:2305.10415, 2023

    2023 arXiv

  33. [41]

    Beyond hallucinations: Enhancing lvlms through hallucination-aware direct preference optimization

    Zhiyuan Zhao, Bin Wang, Linke Ouyang, Xiaoyi Dong, Jiaqi Wang, and Conghui He. Beyond hallucinations: Enhancing lvlms through hallucination-aware direct preference optimization. arXiv preprint arXiv:2311.16839, 2023

    2023 arXiv

  34. [42]

    Llamafactory: Unified efficient fine-tuning of 100+ language models

    Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. Llamafactory: Unified efficient fine-tuning of 100+ language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System ...

    2024

  35. [43]

    R1-zero’s” aha moment” in visual reasoning on a 2b non-sft model, 2025.URL https://arxiv

    Hengguang Zhou, Xirui Li, Ruochen Wang, Minhao Cheng, Tianyi Zhou, and Cho-Jui Hsieh. R1-zero’s” aha moment” in visual reasoning on a 2b non-sft model, 2025.URL https://arxiv. org/abs/2503.05132

    2025 arXiv

  36. [44]

    Aligning modali- ties in vision large language models via preference fine-tuning.arXiv preprint arXiv:2402.11411, 2024

    Yiyang Zhou, Chenhang Cui, Rafael Rafailov, Chelsea Finn, and Huaxiu Yao. Aligning modali- ties in vision large language models via preference fine-tuning.arXiv preprint arXiv:2402.11411, 2024

    2024 arXiv

  37. [45]

    Identify

    Yuxin Zuo, Shang Qu, Yifei Li, Zhangren Chen, Xuekai Zhu, Ermo Hua, Kaiyan Zhang, Ning Ding, and Bowen Zhou. Medxpertqa: Benchmarking expert-level medical reasoning and understanding.arXiv preprint arXiv:2501.18362, 2025. 12 Improving Medical Reasoning with Curriculum-Aware Re...

    2025 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.