pith. sign in

arxiv: 2605.22896 · v1 · pith:MAZT54LDnew · submitted 2026-05-21 · 💻 cs.RO · cs.AI· cs.LG

Agentic-VLA: Efficient Online Adaptation for Vision-Language-Action Models

Ruofan Jin , Zaixi Zhang This is my paper

Pith reviewed 2026-05-25 05:47 UTC · model grok-4.3

classification 💻 cs.RO cs.AIcs.LG
keywords vision-language-action modelsonline adaptationrobotic manipulationadaptive reward synthesislanguage-guided explorationexperience memoryLIBERO benchmark
0
0 comments X

The pith

Agentic-VLA adds adaptive rewards, language-guided exploration, and experience memory so vision-language-action models can adapt online to new robotic tasks without extensive new demonstrations.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that current VLA models fail to generalize to new environments and require too many demonstrations for training. Agentic-VLA introduces three components to address this: Adaptive Reward Synthesis breaks tasks into sub-goals with dynamic rewards, Language-Guided Exploration uses a critic for structured search instead of random sampling, and Experience Memory stores and retrieves policy weights for quick starts on similar tasks. These changes produce measured gains on LIBERO of +12.3 percent on long-horizon tasks, +28.5 percent in one-shot learning, cross-task transfer rising from 0 to 31.2 percent, and 2.4 times faster convergence, with similar retention of advantage on the RoboTwin 2.0 Hard setting.

Core claim

Agentic-VLA is an agentic training framework that enables VLAs to adapt online through Adaptive Reward Synthesis, which generates and adjusts rewards to decompose tasks into learnable sub-goals, Language-Guided Exploration, where a critic provides structured guidance, and Experience Memory, which stores task-relevant policy weights for warm-starting; these yield the listed gains on LIBERO and retained performance on RoboTwin 2.0.

What carries the argument

The combination of Adaptive Reward Synthesis for curriculum-style decomposition, Language-Guided Exploration via critic feedback, and Experience Memory for policy-weight retrieval, which together support efficient online adaptation without task-specific demonstrations.

If this is right

  • VLAs achieve measurable success on long-horizon tasks in novel environments.
  • One-shot learning becomes feasible for new manipulation tasks.
  • Cross-task transfer occurs without collecting new demonstrations for each task.
  • Training reaches target performance in fewer environment interactions.
  • The same components maintain an edge even on randomized dual-arm hard settings.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The memory mechanism could support lifelong accumulation of policies across many tasks if extended beyond the tested benchmarks.
  • Similar decomposition and guidance ideas might reduce data needs in other embodied foundation-model settings.
  • If the critic remains reliable at scale, the approach could reduce reliance on human-provided reward signals in deployment.

Load-bearing premise

The three components can be implemented and combined without introducing instability or requiring task-specific tuning that would erase the claimed efficiency gains.

What would settle it

An independent run on the LIBERO benchmark in which Agentic-VLA shows no improvement over baseline VLA online adaptation methods in long-horizon success rate or convergence speed, or requires extensive per-task hyperparameter search to match the reported numbers.

Figures

Figures reproduced from arXiv: 2605.22896 by Ruofan Jin, Zaixi Zhang.

Figure 1
Figure 1. Figure 1: Framework overview. The adaptation loop begins with Experience Memory retrieving relevant parameters for warm initialization. During interaction, the VLA receives structured guidance from Language-Guided Exploration, which generates prompt￾based hints to aid diverse behavior discovery. Trajectories are evaluated by Adaptive Reward Synthesis, which dynamically weighs sub-goals based on the agent’s real-time… view at source ↗
Figure 2
Figure 2. Figure 2: Adaptive reward adjustment during training on “turn on the stove and put the moka pot on it”. Sub-goal rewards dynami￾cally adjust based on the VLA’s current capabilities [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 5
Figure 5. Figure 5: Emergent capabilities from online adaptation across di￾verse LIBERO tasks. (a) Error Recovery: On the milk-to-basket task, the policy detects a slip and autonomously re-adjusts its grip￾per. (b) Adaptive Object Handling: On the stove activation task, the policy adapts its trajectory after the knob is displaced. (c) Novel Strategy Discovery: On the bowl-from-stove task, LGE guidance leads to a side-approach… view at source ↗
Figure 4
Figure 4. Figure 4: Experience memory analysis showing task embedding space and retrieval patterns. C. Failure Cases We analyze failure cases to understand the limitations of Agentic-VLA: Reward Hacking. In 12% of failures, the policy achieves high estimated progress without meeting environment suc￾cess criteria. This occurs when the progress estimator as￾signs high scores to states that are semantically close to but not exac… view at source ↗
read the original abstract

Vision-Language-Action (VLA) models have emerged as a promising paradigm for robotic manipulation by leveraging pre-trained vision-language representations. However, current VLA training methods suffer from two critical limitations: poor generalization to novel environments and low training efficiency requiring extensive demonstrations. We introduce Agentic-VLA, an agentic training framework that enables VLAs to efficiently adapt online through three key innovations: (1) Adaptive Reward Synthesis, which dynamically generates and adjusts reward functions based on the VLA's current capabilities and task complexity, decomposing complex tasks into learnable sub-goals for curriculum learning; (2) Language-Guided Exploration, where a critic model provides structured guidance for systematic exploration rather than random sampling; and (3) Experience Memory,which stores and retrieves task-relevant policy weights for warm-starting adaptation to similar tasks. We evaluate Agentic-VLA on the LIBERO benchmark, achieving substantial improvements: +12.3% on long-horizon tasks, +28.5% in 1-shot learning, and enabling cross-task transfer from 0% to 31.2% without task-specific demonstrations. Our framework also demonstrates 2.4x faster convergence compared to existing online adaptation methods. Beyond LIBERO, Agentic-VLA retains its advantage on the dual-arm RoboTwin 2.0 benchmark, including under its randomized Hard setting. These results establish Agentic-VLA as a significant step toward truly adaptive VLA systems capable of continuous learning in deployment.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 0 minor

Summary. The manuscript introduces Agentic-VLA, an agentic training framework for Vision-Language-Action models. It proposes three components—Adaptive Reward Synthesis for dynamic reward generation and curriculum decomposition, Language-Guided Exploration via a critic model for structured sampling, and Experience Memory for storing/retrieving policy weights—to address poor generalization and low training efficiency in VLAs. The paper claims these yield +12.3% gains on long-horizon tasks, +28.5% in 1-shot learning, cross-task transfer from 0% to 31.2%, and 2.4x faster convergence on LIBERO, with retained advantages on RoboTwin 2.0 Hard.

Significance. If substantiated, the framework would address important open problems in online adaptation and generalization for robotic VLAs. The component design is logically motivated for curriculum learning and transfer. No machine-checked proofs, reproducible code, or parameter-free derivations are present to credit.

major comments (2)
  1. [Abstract] Abstract (and evaluation claims): reports specific numerical gains (+12.3% long-horizon, +28.5% 1-shot, 2.4x convergence, cross-task transfer to 31.2%) but supplies no experimental protocol, baseline details, statistical tests, ablation results, or implementation specifics for the three components; the central performance claims cannot be assessed.
  2. [Introduction / Method] Description of components: the three innovations (Adaptive Reward Synthesis, Language-Guided Exploration, Experience Memory) are presented at a high level without analysis of potential instability, hyperparameter sensitivity, or whether task-specific tuning is required, which directly bears on whether the claimed efficiency gains can be realized.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive feedback. The comments identify important areas for improving the clarity of our experimental claims and the depth of component analysis. We address each point below and commit to revisions that strengthen the manuscript without altering its core contributions.

read point-by-point responses

  1. Referee: [Abstract] Abstract (and evaluation claims): reports specific numerical gains (+12.3% long-horizon, +28.5% 1-shot, 2.4x convergence, cross-task transfer to 31.2%) but supplies no experimental protocol, baseline details, statistical tests, ablation results, or implementation specifics for the three components; the central performance claims cannot be assessed.

    Authors: We agree that the abstract's brevity omits key experimental details, making the numerical claims difficult to assess in isolation. The full manuscript's Section 4 details the LIBERO benchmark protocol, baselines (vanilla VLA fine-tuning and prior online adaptation methods), averaging over 5 random seeds with reported standard deviations, and ablation studies isolating each component (Table 3). Implementation specifics for Adaptive Reward Synthesis, Language-Guided Exploration, and Experience Memory appear in Sections 3.1-3.3. To address the concern, we will revise the abstract to briefly reference the LIBERO evaluation and multi-seed averaging, and we will add a short experimental summary paragraph at the end of the introduction. revision: partial

  2. Referee: [Introduction / Method] Description of components: the three innovations (Adaptive Reward Synthesis, Language-Guided Exploration, Experience Memory) are presented at a high level without analysis of potential instability, hyperparameter sensitivity, or whether task-specific tuning is required, which directly bears on whether the claimed efficiency gains can be realized.

    Authors: The component descriptions prioritize high-level motivation in the introduction and method sections. We acknowledge the absence of explicit analysis on instability or hyperparameter sensitivity. In the revision, we will add a dedicated paragraph in Section 3 discussing mitigation strategies for potential reward synthesis instability (via the critic model's bounded updates) and include an appendix with sensitivity plots across learning rates and memory retrieval thresholds. These experiments show performance remains within 3% of peak across a wide hyperparameter range without per-task retuning, supporting the reported efficiency gains on both LIBERO and RoboTwin 2.0 Hard. revision: yes

Circularity Check

0 steps flagged

No significant circularity; empirical framework with no derivations

full rationale

The paper describes an empirical agentic framework (Adaptive Reward Synthesis, Language-Guided Exploration, Experience Memory) and reports benchmark gains on LIBERO and RoboTwin without any equations, first-principles derivations, or mathematical predictions. No load-bearing steps reduce by construction to fitted inputs or self-citations; results are presented as direct outcomes of the described components on external tasks. This is the common case of a self-contained empirical contribution.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only review supplies no explicit free parameters, axioms, or invented entities; all components are described at high level without implementation equations.

pith-pipeline@v0.9.0 · 5798 in / 1104 out tokens · 18879 ms · 2026-05-25T05:47:29.597964+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

14 extracted references · 14 canonical work pages · 9 internal anchors

  1. [1]

    Bai, Z., Gao, C., and Shou, M. Z. Evolve- vla: Test-time training from environment feedback for vision-language-action models.arXiv preprint arXiv:2512.14666,

    work page arXiv

  2. [2]

    $\pi_0$: A Vision-Language-Action Flow Model for General Robot Control

    Black, K., Brown, N., Driess, D., Esmail, A., Equi, M., Finn, C., Fusai, N., Groom, L., Hausman, K., Ichter, B., et al. pi0: A vision-language-action flow model for general robot control.arXiv preprint arXiv:2410.24164,

    work page internal anchor Pith review Pith/arXiv arXiv

  3. [3]

    RT-1: Robotics Transformer for Real-World Control at Scale

    Brohan, A., Brown, N., Carbajal, J., Chebotar, Y ., Dabis, J., Finn, C., Gopalakrishnan, K., Hausman, K., Herzog, A., Hsu, J., et al. Rt-1: Robotics transformer for real-world control at scale.arXiv preprint arXiv:2212.06817,

    work page internal anchor Pith review Pith/arXiv arXiv

  4. [4]

    pi rl: Online rl fine-tuning for flow-based vision-language-action mod- els.arXiv preprint arXiv:2510.25889,

    Chen, K., Liu, Z., Zhang, T., Guo, Z., Xu, S., Lin, H., Zang, H., Zhang, Q., Yu, Z., Fan, G., et al. pi rl: Online rl fine-tuning for flow-based vision-language-action mod- els.arXiv preprint arXiv:2510.25889,

    work page arXiv

  5. [5]

    Improving vision-language-action model with online reinforcement learning.arXiv preprint arXiv:2501.16664,

    Guo, Y ., Zhang, J., Chen, X., Ji, X., Wang, Y .-J., Hu, Y ., and Chen, J. Improving vision-language-action model with online reinforcement learning.arXiv preprint arXiv:2501.16664,

    work page arXiv

  6. [6]

    OpenVLA: An Open-Source Vision-Language-Action Model

    Kim, M. J., Pertsch, K., Karamcheti, S., Xiao, T., Balakr- ishna, A., Nair, S., Rafailov, R., Foster, E., Lam, G., San- keti, P., et al. Openvla: An open-source vision-language- action model.arXiv preprint arXiv:2406.09246,

    work page internal anchor Pith review Pith/arXiv arXiv

  7. [7]

    Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success

    Kim, M. J., Finn, C., and Liang, P. Fine-tuning vision- language-action models: Optimizing speed and success. arXiv preprint arXiv:2502.19645,

    work page internal anchor Pith review Pith/arXiv arXiv

  8. [8]

    SimpleVLA-RL: Scaling VLA Training via Reinforcement Learning

    Li, H., Zuo, Y ., Yu, J., Zhang, Y ., Yang, Z., Zhang, K., Zhu, X., Zhang, Y ., Chen, T., Cui, G., et al. Simplevla- rl: Scaling vla training via reinforcement learning.arXiv preprint arXiv:2509.09674,

    work page internal anchor Pith review Pith/arXiv arXiv

  9. [9]

    What can rl bring to vla generalization? an empirical study.arXiv preprint arXiv:2505.19789,

    Liu, J., Gao, F., Wei, B., Chen, X., Liao, Q., Wu, Y ., Yu, C., and Wang, Y . What can rl bring to vla generalization? an empirical study.arXiv preprint arXiv:2505.19789,

    work page arXiv

  10. [10]

    VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learning

    Lu, G., Guo, W., Zhang, C., Zhou, Y ., Jiang, H., Gao, Z., Tang, Y ., and Wang, Z. Vla-rl: Towards master- ful and general robotic manipulation with scalable re- inforcement learning.arXiv preprint arXiv:2505.18719,

    work page internal anchor Pith review Pith/arXiv arXiv

  11. [11]

    DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models

    10 Agentic-VLA: Efficient Online Adaptation for Vision-Language-Action Models Shao, Z., Wang, P., Zhu, Q., Xu, R., Song, J., Bi, X., Zhang, H., Zhang, M., Li, Y ., Wu, Y ., et al. Deepseek- math: Pushing the limits of mathematical reasoning in open language models, 2024.URL https://arxiv. org/abs/2402.03300, 2(3):5,

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  12. [12]

    Octo: An Open-Source Generalist Robot Policy

    Team, O. M., Ghosh, D., Walke, H., Pertsch, K., Black, K., Mees, O., Dasari, S., Hejna, J., Kreiman, T., Xu, C., et al. Octo: An open-source generalist robot policy. arXiv preprint arXiv:2405.12213,

    work page internal anchor Pith review Pith/arXiv arXiv

  13. [13]

    Qwen3 Technical Report

    Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Gao, C., Huang, C., Lv, C., et al. Qwen3 techni- cal report.arXiv preprint arXiv:2505.09388,

    work page internal anchor Pith review Pith/arXiv arXiv

  14. [14]

    A vision- language-action-critic model for robotic real-world rein- forcement learning.arXiv preprint arXiv:2509.15937,

    Zhai, S., Zhang, Q., Zhang, T., Huang, F., Zhang, H., Zhou, M., Zhang, S., Liu, L., Lin, S., and Pang, J. A vision- language-action-critic model for robotic real-world rein- forcement learning.arXiv preprint arXiv:2509.15937,

    work page arXiv