REVIEW 11 cited by
GRAPE: Generalizing Robot Policy via Preference Alignment
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Despite the recent advancements of vision-language-action (VLA) models on a variety of robotics tasks, they suffer from critical issues such as poor generalizability to unseen tasks, due to their reliance on behavior cloning exclusively from successful rollouts. Furthermore, they are typically fine-tuned to replicate demonstrations collected by experts under different settings, thus introducing distribution bias and limiting their adaptability to diverse manipulation objectives, such as efficiency, safety, and task completion. To bridge this gap, we introduce GRAPE: Generalizing Robot Policy via Preference Alignment. Specifically, GRAPE aligns VLAs on a trajectory level and implicitly models reward from both successful and failure trials to boost generalizability to diverse tasks. Moreover, GRAPE breaks down complex manipulation tasks to independent stages and automatically guides preference modeling through customized spatiotemporal constraints with keypoints proposed by a large vision-language model. Notably, these constraints are flexible and can be customized to align the model with varying objectives, such as safety, efficiency, or task success. We evaluate GRAPE across a diverse array of tasks in both real-world and simulated environments. Experimental results demonstrate that GRAPE enhances the performance of state-of-the-art VLA models, increasing success rates on in-domain and unseen manipulation tasks by 51.79% and 58.20%, respectively. Additionally, GRAPE can be aligned with various objectives, such as safety and efficiency, reducing collision rates by 37.44% and rollout step-length by 11.15%, respectively. All code, models, and data are available at https://grape-vla.github.io/
Forward citations
Cited by 11 Pith papers
-
Freeform Preference Learning for Robotic Manipulation
Language-conditioned multi-axis human preferences yield denser rewards and steerable robot policies that outperform sparse and binary-preference baselines by 38 points on long-horizon manipulation.
-
SOLE-R1: Video-Language Reasoning as the Sole Reward for On-Robot Reinforcement Learning
A video-language model with per-timestep spatiotemporal CoT and dense progress prediction can serve as the sole reward for zero-shot online robot RL on 24 unseen manipulation tasks.
-
RedFlow: Redirect Failure into Action-Level Corrections for Flow-matching VLA Policy
RedFlow improves flow-matching VLA policies offline by labeling individual actions from failed rollouts with progress signals and pushing those actions toward successful alternatives found in similar contexts.
-
RL$^2$-VLA: Adaptive RL Latent Compositional Steering with Test-Time Scaling for Vision-Language-Action Models
RL² improves VLA robot success rates by conditionally composing an offline RL policy's actions with the frozen VLA only when a failure detector flags impending failure.
-
Hy-Embodied-0.5-VLA: From Vision-Language-Action Models to a Real-World Robot Learning Stack
A full learning stack—10K-hour UMI data, a flow-matching VLA, preference-optimization RL, and asynchronous deployment—reports SOTA RoboTwin results and cross-embodiment transfer to four real robots.
-
VLA-Arena: An Open-Source Framework for Benchmarking Vision-Language-Action Models
An open benchmark with 170 graded manipulation tasks shows current VLA robot policies memorize their training settings, degrade sharply under visual shifts, ignore safety constraints, and fail to compose long-horizon skills.
-
Reinforcement Fine-Tuning of Flow-Matching Policies for Vision-Language-Action Models
FPO fine-tunes flow-matching vision-language-action policies with a PPO-style objective that replaces intractable policy ratios with per-sample conditional flow-matching loss differences, reaching 87.2% average succes...
-
Adversarial Attacks on Robotic Vision Language Action Models
Text-based adversarial suffixes can make OpenVLA robot policies elicit chosen target actions with over 90% success on one-hot targets and persist across rollout steps.
-
Beyond Flat Policies: Hierarchical Post-Training for Embodied Agents in Robotic Manipulation
HiRoC uses a language planner to split manipulation tasks into subgoals and a reinforcement-learned executor to follow them, reporting 93.5% average success on LIBERO.
-
Balancing Signal and Variance: Adaptive Offline RL Post-Training for VLA Flow Models
ARFM adaptively adjusts a scaling factor in the flow-matching loss so that offline RL advantage signals are preserved while gradient variance is controlled, improving VLA robot policy fine-tuning.
-
Continual Learning for Generative AI: From LLMs to MLLMs and Beyond
A survey that categorizes continual learning methods for generative models into architecture-based, regularization-based, and replay-based paradigms across four model families.
Discussion (0). Continue with ORCID to comment.