Pith. sign in

REVIEW 4 cited by

Vision-Language Models as a Source of Rewards

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2312.09187 v3 pith:PRWSEKXD submitted 2023-12-14 cs.LG

Vision-Language Models as a Source of Rewards

Kate Baumli , Satinder Baveja , Feryal Behbahani , Harris Chan , Gheorghe Comanici , Sebastian Flennerhag , Maxime Gazeau , Kristian Holsheimer
show 19 more authors
Dan Horgan Michael Laskin Clare Lyle Hussain Masoom Kay McKinney Volodymyr Mnih Alexander Neitz Dmitry Nikulin Fabio Pardo Jack Parker-Holder John Quan Tim Rockt\"aschel Himanshu Sahni Tom Schaul Yannick Schroecker Stephen Spencer Richie Steigerwald Luyu Wang Lei Zhang
This is my paper
classification cs.LG
keywords agentsgoalsrewardsmodelsvisualachievementbuildinggeneralist
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved
0 comments
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Building generalist agents that can accomplish many goals in rich open-ended environments is one of the research frontiers for reinforcement learning. A key limiting factor for building generalist agents with RL has been the need for a large number of reward functions for achieving different goals. We investigate the feasibility of using off-the-shelf vision-language models, or VLMs, as sources of rewards for reinforcement learning agents. We show how rewards for visual achievement of a variety of language goals can be derived from the CLIP family of models, and used to train RL agents that can achieve a variety of language goals. We showcase this approach in two distinct visual domains and present a scaling trend showing how larger VLMs lead to more accurate rewards for visual goal achievement, which in turn produces more capable RL agents.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. QVal: Cheaply Evaluating Dense Supervision Signals for Long-Horizon LLM Agents

    cs.LG 2026-06 unverdicted novelty 7.0

    QVal is a new evaluation framework that directly measures dense supervision quality via Q-alignment to a reference policy, showing simple prompting baselines outperform 21 other methods across environments and models.

  2. SOLE-R1: Video-Language Reasoning as the Sole Reward for On-Robot Reinforcement Learning

    cs.RO 2026-03 conditional novelty 6.5

    A video-language model with per-timestep spatiotemporal CoT and dense progress prediction can serve as the sole reward for zero-shot online robot RL on 24 unseen manipulation tasks.

  3. Learning Process Rewards via Success Visitation Matching for Efficient RL

    cs.LG 2026-06 unverdicted novelty 6.0

    Success Visitation Matching uses a discriminator to turn sparse outcome rewards into dense process rewards by matching visitations of successful episodes, provably preserving the optimal policy and speeding up robotic...

  4. Progress Reward Modeling for Robotic Learning: A Comprehensive Survey

    cs.RO 2026-07 conditional novelty 5.0

    A survey organizing progress reward modeling for robot learning into interface, method, and data/evaluation layers, with a four-family method taxonomy and a cautionary split between progress fidelity and downstream utility.