REVIEW 14 cited by
Vision-R1: Evolving Human-Free Alignment in Large Vision-Language Models via Vision-Guided Reinforcement Learning
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Large Vision-Language Models (LVLMs) typically follow a two-stage training paradigm-pretraining and supervised fine-tuning. Recently, preference optimization, derived from the language domain, has emerged as an effective post-training reinforcement strategy to enhance capabilities of LVLMs. However, constructing high-quality human-annotated preference data and developing robust reward models to mimic these preferences are both costly and challenging. Motivated by this observation, we propose Vision-R1, a novel vision-guided R1-like reinforcement learning algorithm for LVLMs that rewards models with definitive vision feedback. It only leverages curated instruction data, eliminating the need for specialized reward models and handcrafted preference datasets. We incorporate a criterion-driven reward function that further integrates multi-dimensional feedback to evaluate model completions comprehensively based on the vision task logic. Furthermore, we introduce a progressive rule refinement strategy that dynamically adjusts the reward criteria during training, enabling continuous model improvement and mitigating reward hacking. Extensive experiments on both in-distribution and out-of-distribution benchmarks demonstrate that fine-tuning the 7B LVLMs with Vision-R1 achieves consistent performance gains, with even up to 50% improvement and surpassing the state-of-the-art 10x size model.
Forward citations
Cited by 14 Pith papers
-
REVA-PO: Stabilizing Reinforcement Learning for Chest X-ray Report Generation
REVA-PO stabilizes GRPO-style RL for CXR report generation via response-level adaptive KL weights and validation-anchored policy resets, reporting new SOTA BLEU and clinical F1 scores.
-
Surgery-R1: Advancing Surgical-VQLA with Reasoning Multimodal Large Language Model via Reinforcement Learning
Surgery-R1 uses supervised fine-tuning and reinforcement fine-tuning to give a surgical visual question answering model chain-of-thought reasoning, improving accuracy and localization on two EndoVis benchmarks.
-
Reinforcement Learning Tuning for VideoLLMs: Reward Design and Data Efficiency
A GRPO-based post-training recipe for video LLMs using discrete QA rewards plus continuous temporal IoU rewards with variance-based data selection outperforms SFT and Video-R1.
-
Mixed-R1: Unified Reward Perspective For Reasoning Capability in Multimodal Large Language Models
Mixed-R1 uses four reward types under GRPO, including a new bidirectional max-average token similarity (BMAS) reward, and lifts MLLM reasoning benchmarks by 2-5%.
-
Vad-R1: Towards Video Anomaly Reasoning via Perception-to-Cognition Chain-of-Thought
A two-stage trained MLLM with a perception-to-cognition chain-of-thought and a self-verification RL reward outperforms prior models on video anomaly detection and reasoning.
-
Align and Surpass Human Camouflaged Perception: Visual Refocus Reinforcement Fine-Tuning
A curriculum reinforcement-learning framework with in-context refocus examples improves camouflaged object classification and detection for a vision-language model, and the authors report surpassing human performance ...
-
Unveiling the Compositional Ability Gap in Vision-Language Reasoning Model
RL-trained VLMs generalize compositionally far better than SFT-trained ones on synthetic geometry and spatial tasks, but cross-modal combination remains weak, and a caption-before-thinking plus progress-reward recipe ...
-
R1-ShareVL: Incentivizing Reasoning Capability of Multimodal Large Language Models via Share-GRPO
Share-GRPO creates paraphrased and visually augmented versions of reasoning questions and shares answers and reward signals across versions, improving multimodal reasoning without cold-start SFT.
-
VisRAG2.0: Mitigating Visual Hallucinations via Evidence-Guided Multi-Image Reasoning in Visual Retrieval-Augmented Generation
EVisRAG, an evidence-guided observe-record-reason-answer pipeline trained with reward-scoped GRPO, improves multi-image visual QA accuracy by about 19% over its backbone VLM across five benchmarks.
-
Reinforced Visual Perception with Tools
ReVPT uses GRPO reinforcement learning with a cold-start SFT phase to make Qwen2.5-VL models call visual tools, improving perception benchmarks over SFT and text-only RL baselines.
-
Reinforcing VLMs to Use Tools for Detailed Visual Reasoning Under Resource Constraints
A 3B VLM trained with GRPO to call a zoom tool improves V*Bench accuracy by 5.7% over its base model but degrades TextVQA and HR-Bench performance.
-
Reinforcing Video Reasoning with Focused Thinking
A GRPO variant with token-level KL weighting and partial-credit rewards improves video-QA on modified multi-answer benchmarks, but transfer to original single-answer benchmarks is not established.
-
An Empirical Study on Reinforcement Learning for Reasoning-Search Interleaved LLM Agents
Format rewards help base LLMs more than intermediate retrieval rewards, general-purpose backbones beat reasoning-distilled ones, and stronger search engines stabilize RL training for LLM search agents.
-
Aligning Large Vision-Language Models by Deep Reinforcement Learning and Direct Preference Optimization
A survey categorizing deep reinforcement learning and direct preference optimization methods for aligning large vision-language models, with tables of studies and datasets and no new experimental result.
Discussion (0). Sign in to comment.