REVIEW 22 cited by
RL-VLM-F: Reinforcement Learning from Vision Language Foundation Model Feedback
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Reward engineering has long been a challenge in Reinforcement Learning (RL) research, as it often requires extensive human effort and iterative processes of trial-and-error to design effective reward functions. In this paper, we propose RL-VLM-F, a method that automatically generates reward functions for agents to learn new tasks, using only a text description of the task goal and the agent's visual observations, by leveraging feedbacks from vision language foundation models (VLMs). The key to our approach is to query these models to give preferences over pairs of the agent's image observations based on the text description of the task goal, and then learn a reward function from the preference labels, rather than directly prompting these models to output a raw reward score, which can be noisy and inconsistent. We demonstrate that RL-VLM-F successfully produces effective rewards and policies across various domains - including classic control, as well as manipulation of rigid, articulated, and deformable objects - without the need for human supervision, outperforming prior methods that use large pretrained models for reward generation under the same assumptions. Videos can be found on our project website: https://rlvlmf2024.github.io/
Forward citations
Cited by 22 Pith papers
-
Prompt-Driven Exploration
Prompt-Driven Exploration refines language prompts from rollout videos via a VLM, enabling RL to escape zero-reward VLA and LLM policies where action-space noise fails.
-
Freeform Preference Learning for Robotic Manipulation
Language-conditioned multi-axis human preferences yield denser rewards and steerable robot policies that outperform sparse and binary-preference baselines by 38 points on long-horizon manipulation.
-
SOLE-R1: Video-Language Reasoning as the Sole Reward for On-Robot Reinforcement Learning
A video-language model with per-timestep spatiotemporal CoT and dense progress prediction can serve as the sole reward for zero-shot online robot RL on 24 unseen manipulation tasks.
-
Weights or Skills? A Survey of Robot-Learning Techniques: from Action-Predicting Weights to Robots that Write their Own Skills
A taxonomy of robot learning on a weights-versus-skills axis, with a five-rung self-improvement ladder whose top cell (feedback plus memory plus search) holds only a few recent systems.
-
MARS-RA: Rank Aggregation for Credit Assignment via Multimodal Comparisons in Embodied Multi-Agent Cooperation
Credit assignment via LMM pairwise comparisons plus Bradley–Terry rank aggregation and potential-based shaping improves cooperative MARL under sparse rewards and dynamic agent counts.
-
Learning More from Less: Reinforcement Learning from Hindsight
Hindsight language-and-reward relabeling of failed VLA rollouts by a VLM yields roughly 5× sample-efficiency gains over standard GRPO on OOD LIBERO-PRO and real Franka tasks.
-
HALO: Human Preference Aligned Offline Reward Learning for Robot Navigation
HALO learns a vision-based navigation reward from human preference rankings on egocentric video, and an IQL policy using it beats several baselines in 10-trial real-world tests.
-
ROAD: Responsibility-Oriented Reward Design for Reinforcement Learning in Autonomous Driving
A responsibility-aware crash penalty, built from a traffic-law knowledge graph and a vision-language blame classifier, improves MetaDrive success rates and shifts reported collision blame away from the ego vehicle.
-
ManiTaskGen: A Comprehensive Task Generator for Benchmarking and Improving Vision-Language Agents on Embodied Decision-Making
ManiTaskGen automatically generates diverse, feasible mobile manipulation tasks from any input scene, and uses them to benchmark and improve vision-language robot agents.
-
PRISM: Projection-based Reward Integration for Scene-Aware Real-to-Sim-to-Real Transfer with Few Demonstrations
Using five demonstrations, PRISM builds a simulator and trains a policy with a vision-language-model reward, reaching 82% success under randomized conditions on six tabletop tasks.
-
Contrastive Learning from Exploratory Actions: Leveraging Natural Interactions for Preference Elicitation
Features learned from users' exploratory clicks on robot behaviors outperform self-supervised features for preference elicitation across visual, auditory, and kinetic signals.
-
SILMM: Self-Improving Large Multimodal Models for Compositional Text-to-Image Generation
An iterative self-feedback loop with discrete DPO and kernel-based continuous DPO improves compositional text-to-image alignment of LMMs by double-digit percentages without human labels.
-
MALMM: Multi-Agent Large Language Models for Zero-Shot Robotics Manipulation
MALMM, a three-agent LLM framework with per-step environment feedback, achieves 81% average success on nine RLBench tasks versus 50% for a single-agent LLM baseline, in zero-shot settings.
-
LLM-Based Offline Learning for Embodied Agents via Consistency-Guided Reward Ensemble
CoREN uses an LLM offline to estimate dense action rewards, filters them through three consistency checks, and aligns them to sparse success labels to train a small offline RL agent for household instruction-following tasks.
-
ViSTa Dataset: Do vision-language models understand sequential tasks?
ViSTa is a new hierarchical video benchmark showing that vision-language models recognize objects well but fail to understand action order in sequential tasks.
-
LLM-as-a-Verifier: A General-Purpose Verification Framework
Expecting over scoring-token logits yields continuous, scalable verification that improves agent trajectory selection and dense RL rewards across coding, robotics, and medical benchmarks.
-
Sample Efficient Reinforcement Learning via Large Vision Language Model Distillation
Showing that an LVLM teacher's soft action probabilities, blended into the RL loss, speed up MiniGrid agents by about 2.5x in sample efficiency.
-
TREND: Tri-teaching for Robust Preference-based Reinforcement Learning with Demonstrations
TREND combines cyclic small-loss selection among three reward models with few-shot demonstration pretraining and regularization, yielding robust policies on Meta-World tasks under up to 40 percent label noise.
-
FDPP: Fine-tune Diffusion Policy with Human Preference
FDPP aligns a pre-trained diffusion policy with human preferences by training a reward model from preference labels and fine-tuning the policy with RL plus KL regularization.
-
VLM-RL: A Unified Vision Language Models and Reinforcement Learning Framework for Safe Autonomous Driving
VLM-RL uses CLIP-based contrasting positive and negative language goals plus vehicle state rewards to train RL agents for driving in CARLA, claiming safer and more efficient policies.
-
CLIP-RLDrive: Human-Aligned Autonomous Driving via CLIP-Based Reward Shaping in Reinforcement Learning
In a simulated unsignalized intersection, CLIP-based reward shaping improves DQN success to 96%, but it degrades PPO and the evaluation lacks seeds, error bars, and released code.
-
YingSound: Video-Guided Sound Effects Generation with Multi-modal Chain-of-Thought Controls
A video-guided sound effects model with a learnable audio-visual aggregator and multi-modal chain-of-thought module reports strong VGGSound benchmark scores, but its few-shot claim rests on three qualitative samples.
Discussion (0). Continue with ORCID to comment.