Pith. sign in

REVIEW 28 cited by

Improving Vision-Language-Action Model with Online Reinforcement Learning

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2501.16664 v1 pith:KHJIRD3M submitted 2025-01-28 cs.RO cs.CVcs.LG

Yanjiang Guo , Jianke Zhang , Xiaoyu Chen , Xiang Ji , Yen-Jen Wang , Yucheng Hu , Jianyu Chen This is my paper
classification cs.ROcs.CVcs.LG
keywords modelslargelearningimprovereinforcementsupervisedchallengesfine-tuning
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Recent studies have successfully integrated large vision-language models (VLMs) into low-level robotic control by supervised fine-tuning (SFT) with expert robotic datasets, resulting in what we term vision-language-action (VLA) models. Although the VLA models are powerful, how to improve these large models during interaction with environments remains an open question. In this paper, we explore how to further improve these VLA models via Reinforcement Learning (RL), a commonly used fine-tuning technique for large models. However, we find that directly applying online RL to large VLA models presents significant challenges, including training instability that severely impacts the performance of large models, and computing burdens that exceed the capabilities of most local machines. To address these challenges, we propose iRe-VLA framework, which iterates between Reinforcement Learning and Supervised Learning to effectively improve VLA models, leveraging the exploratory benefits of RL while maintaining the stability of supervised learning. Experiments in two simulated benchmarks and a real-world manipulation suite validate the effectiveness of our method.

Discussion (0). Sign in to comment.

Forward citations

Cited by 28 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Adapting Generalist Robot Policies with Semantic Reinforcement Learning

    cs.RO 2026-06 unverdicted novelty 7.0 of 10

    SARL optimizes language prompt inputs to generalist vision-language-action policies through online RL to solve complex long-horizon tasks by composing existing skills.

  2. Learn Where Outcomes Diverge: Efficient VLA RL via Probabilistic Chunk Masking

    cs.LG 2026-05 unverdicted novelty 7.0 of 10

    PCM uses success-failure action variance to probabilistically select and mask chunks for gradient updates in GRPO, matching standard success rates with 2.38x wall-clock speedup and 60% lower memory on LIBERO benchmarks.

  3. PhySe-RPO: Physics and Semantics Guided Relative Policy Optimization for Diffusion-Based Surgical Smoke Removal

    cs.AI 2026-03 unverdicted novelty 7.0 of 10

    PhySe-RPO enables diffusion-based surgical smoke removal by converting restoration into a stochastic policy optimized with physics consistency and CLIP semantic rewards under limited supervision.

  4. VLM4VLA: Revisiting Vision-Language-Models in Vision-Language-Action Models

    cs.CV 2026-01 conditional novelty 7.0 of 10

    Using a simple action-token adapter, nine VLMs are compared as robot policy backbones, showing general VLM ability transfers poorly to control and the vision encoder is the key bottleneck.

  5. RL Bootstrapping of OpenVLA-OFT for a Novel Robot Embodiment

    cs.RO 2026-08 conditional novelty 6.0 of 10

    RL-only fine-tuning with dense geometric rewards aligns OpenVLA-OFT to a novel cable-driven robot, reaching 53.5% average success on four directional commands without any demonstration data.

  6. Learning Process Rewards via Success Visitation Matching for Efficient RL

    cs.LG 2026-06 unverdicted novelty 6.0 of 10

    Success Visitation Matching uses a discriminator to turn sparse outcome rewards into dense process rewards by matching visitations of successful episodes, provably preserving the optimal policy and speeding up robotic...

  7. Agentic-VLA: Efficient Online Adaptation for Vision-Language-Action Models

    cs.RO 2026-05 unverdicted novelty 6.0 of 10

    Agentic-VLA enables efficient online adaptation of VLA models, delivering +12.3% on long-horizon tasks, +28.5% in 1-shot learning, and 2.4x faster convergence on LIBERO through three new components.

  8. E-VLA: Event-Augmented Vision-Language-Action Model for Dark and Blurred Scenes

    cs.CV 2026-04 conditional novelty 6.0 of 10

    E-VLA integrates event streams directly into VLA models via lightweight fusion, raising Pick-Place success from 0% to 60-90% at 20 lux and from 0% to 20-25% under severe motion blur.

  9. Veo-Act: How Far Can Frontier Video Models Advance Generalizable Robot Manipulation?

    cs.RO 2026-04 unverdicted novelty 6.0 of 10

    Veo-3 video predictions enable approximate task-level robot trajectories in zero-shot settings but require hierarchical integration with low-level VLA policies for reliable manipulation performance.

  10. Towards Long-Lived Robots: Continual Learning VLA Models via Reinforcement Fine-Tuning

    cs.RO 2026-02 unverdicted novelty 6.0 of 10

    LifeLong-RFT applies chunking-level on-policy reinforcement learning with Quantized Action Consistency Reward, Continuous Trajectory Alignment Reward, and Format Compliance Reward to fine-tune VLA models, achieving a ...

  11. TwinRL: Digital Twin-Driven Reinforcement Learning for Real-World Robotic Manipulation

    cs.RO 2026-02 unverdicted novelty 6.0 of 10

    TwinRL expands RL exploration via digital twin reconstruction and twin RL warm-up to guide real-world learning, reaching near-100% success with 20 minutes of on-robot time across four tasks.

  12. VLA-Arena: An Open-Source Framework for Benchmarking Vision-Language-Action Models

    cs.RO 2025-12 conditional novelty 6.0 of 10

    An open benchmark with 170 graded manipulation tasks shows current VLA robot policies memorize their training settings, degrade sharply under visual shifts, ignore safety constraints, and fail to compose long-horizon skills.

  13. $\pi^{*}_{0.6}$: a VLA That Learns From Experience

    cs.LG 2025-11 unverdicted novelty 6.0 of 10

    RECAP enables a generalist VLA to self-improve via advantage-conditioned RL on mixed real-world data, more than doubling throughput and halving failure rates on hard manipulation tasks.

  14. Ctrl-World: A Controllable Generative World Model for Robot Manipulation

    cs.RO 2025-10 unverdicted novelty 6.0 of 10

    A controllable world model trained on the DROID dataset generates consistent multi-view robot trajectories for over 20 seconds and improves generalist policy success rates by 44.7% via imagined trajectory fine-tuning.

  15. Reinforcement Fine-Tuning of Flow-Matching Policies for Vision-Language-Action Models

    cs.LG 2025-10 conditional novelty 6.0 of 10

    FPO fine-tunes flow-matching vision-language-action policies with a PPO-style objective that replaces intractable policy ratios with per-sample conditional flow-matching loss differences, reaching 87.2% average succes...

  16. SimpleVLA-RL: Scaling VLA Training via Reinforcement Learning

    cs.RO 2025-09 conditional novelty 6.0 of 10

    SimpleVLA-RL applies tailored reinforcement learning to VLA models, reaching SoTA on LIBERO, outperforming π₀ on RoboTwin, and surpassing SFT in real-world tasks while reducing data needs and identifying a 'pushcut' p...

  17. VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learning

    cs.RO 2025-05 conditional novelty 6.0 of 10

    VLA-RL applies online RL to pretrained VLAs, yielding a 4.5% gain over strong baselines on 40 LIBERO manipulation tasks and matching commercial models like π₀-FAST.

  18. DexVLA: Vision-Language Model with Plug-In Diffusion Expert for General Robot Control

    cs.RO 2025-02 unverdicted novelty 6.0 of 10

    DexVLA combines a scaled diffusion action expert with embodiment curriculum learning to achieve better generalization and performance than prior VLA models on diverse robot hardware and long-horizon tasks.

  19. Video Prediction Policy: A Generalist Robot Policy with Predictive Visual Representations

    cs.CV 2024-12 unverdicted novelty 6.0 of 10

    Video Prediction Policy conditions robot action learning on future-frame predictions inside fine-tuned video diffusion models, yielding 18.6% relative gains on Calvin ABC-D and 31.6% higher real-world success rates.

  20. AllDayNav: Lifelong Navigation via Real-World Reinforcement Learning

    cs.RO 2026-06 unverdicted novelty 5.0 of 10

    AllDayNav encodes scene dynamics into a large model's parameters via RL and a multimodal memory, achieving near-100% success rates in lifelong navigation and outperforming map-based and VLM baselines.

  21. DyGRO-VLA: Cross-Task Scaling of Vision-Language-Action Models via Dynamic Grouped Residual Optimization

    cs.RO 2026-05 unverdicted novelty 5.0 of 10

    DyGRO-VLA is a two-stage optimization framework for cross-task scaling of Vision-Language-Action models via dynamic grouped residual optimization in RL.

  22. E-VLA: Event-Augmented Vision-Language-Action Model for Dark and Blurred Scenes

    cs.CV 2026-04 conditional novelty 5.0 of 10

    Degradation-Driven Prompting improves VLM accuracy on deceptive visual tasks by downsampling inputs and routing them through classification, external visual tools, and a critic.

  23. VGAS: Value-Guided Action-Chunk Selection for Few-Shot Vision-Language-Action Adaptation

    cs.AI 2026-02 unverdicted novelty 5.0 of 10

    VGAS uses best-of-N selection with a geometrically grounded critic and explicit regularization to improve success rates of few-shot VLA policies under limited data and distribution shifts.

  24. Reflection-Based Task Adaptation for Self-Improving VLA

    cs.RO 2025-10 unverdicted novelty 5.0 of 10

    Reflective Self-Adaptation combines failure-reflective reinforcement learning with success-guided imitation learning to enable faster and more reliable task adaptation for pre-trained Vision-Language-Action models.

  25. Balancing Signal and Variance: Adaptive Offline RL Post-Training for VLA Flow Models

    cs.RO 2025-09 conditional novelty 5.0 of 10

    ARFM adaptively adjusts a scaling factor in the flow-matching loss so that offline RL advantage signals are preserved while gradient variance is controlled, improving VLA robot policy fine-tuning.

  26. Source Component Shift Adaptation via Offline Decomposition and Online Mixing Approach

    cs.LG 2025-08 unverdicted novelty 5.0 of 10

    The claimed result is that source-component-shift adaptation splits cleanly into offline component learning via EM and online mixing-weight updates, cutting cumulative test loss by up to 67.4%.

  27. Leveraging OS-Level Primitives for Robotic Action Management

    cs.OS 2025-08 conditional novelty 4.0 of 10

    Applying OS-style exception handling, context caching, and replay to robotic action slices raises success rates 7x to 24x and cuts execution steps up to 74% for repetitive manipulation tasks, without retraining the VLA model.

  28. Aligning Large Vision-Language Models by Deep Reinforcement Learning and Direct Preference Optimization

    cs.LG 2025-09 unverdicted

    A survey categorizing deep reinforcement learning and direct preference optimization methods for aligning large vision-language models, with tables of studies and datasets and no new experimental result.

Pith tools