Pith. sign in

REVIEW 4 major objections 5 minor 145 references

This paper argues that test-time action diversity from reinforcement-learned steering helps only when the base robot policy is failing, and that gating steering on a failure detector yields gains up to +17.3% on out-of-domain tasks.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 10:18 UTC pith:GADTS5BF

load-bearing objection A practical, well-built adaptive steering method whose 'scaling law' foundation is thinner than it looks — worth refereeing, but the central asymmetry claim needs direct validation. the 4 major comments →

arxiv 2607.26991 v2 pith:GADTS5BF submitted 2026-07-29 cs.RO

RL²-VLA: Adaptive RL Latent Compositional Steering with Test-Time Scaling for Vision-Language-Action Models

Derek Ming Siang Tan , Shailesh Shailesh , Srikrishna Iyer , William Wei Jie Teo , Yuanliang Ju , Qiao Gu , Guillaume Sartoretti This is my paper
classification cs.RO
keywords vision-language-action modelsoffline reinforcement learningflow matchingtest-time scalingadaptive inferencefailure detectionconformal predictionrobot manipulation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper asks when and how to steer a frozen vision-language-action (VLA) model at inference time. It argues that sampling more actions from the same policy is not enough, because those samples inherit the base policy's failure modes. Its RL² framework trains a lightweight offline RL policy on the VLA's internal feature vectors, composes the RL policy's flow velocity with the frozen VLA's flow velocity to generate diverse action candidates, and enables that composition only when a learned failure detector says failure is likely. The supporting discovery is a state-dependent scaling law: under an oracle verifier, RL compositional steering reduces action error on failure tuples but degrades already-accurate actions on success tuples. In the paper's experiments, adaptive steering beats both the base VLA and always-on steering, with success-rate gains up to +17.3% in out-of-domain simulation and +17.5% on real hardware.

Core claim

The central claim is a pair of test-time scaling laws. With an oracle verifier, generating more action candidates from a flow-matching VLA reduces normalized action error as a power law in the number of samples. The law is not uniform, however: RL-based compositional steering is the strongest reducer of action error on failure tuples and among the worst on success tuples, where it unnecessarily perturbs accurate actions. The paper reads this asymmetry as evidence that inference-time steering should be adaptive—switched on only when the base policy is predicted to fail—and reports that this adaptive rule outperforms both the base policy and always-on steering across two simulation benchmarks,

What carries the argument

Compositional steering of flow velocities. In a flow-matching VLA, actions are generated by following a velocity field that carries noise onto action trajectories. The paper combines the frozen VLA's velocity field with that of a lightweight offline RL steering policy—both conditioned on latents (internal feature vectors) from the VLA action expert—as a weighted average, with the weight drawn from a Gaussian. This composes the imitation prior with RL-induced diversity. The RL policy is trained offline with an adjoint-matching Q-learning objective. A separate lightweight failure detector, calibrated by conformal prediction, decides when to activate the composition; an external verifier ranks

Load-bearing premise

The load-bearing assumption is that the states where the base policy is about to fail can be recognized at run time: the scaling law is derived with an oracle that knows the ground-truth action, while deployment relies on a learned failure detector making the same distinction.

What would settle it

On an out-of-domain task with success/failure defined by task completion, compare always-on composition, adaptive composition, and the base policy using the same verifier. If a perfect failure oracle does not outperform always-on steering, or if compositional steering reduces action error on success states when failure labels come from task outcome rather than action NRMSE, the central asymmetry claim is refuted.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Adaptive steering beats always-on steering in the paper's experiments, implying that fixed test-time intervention wastes its benefit on states where the base policy already succeeds.
  • Latent-conditioned RL steering outperforms behavior-cloned steering and raw-observation-conditioned steering, implying that internal VLA features carry the state information needed for recovery.
  • Scaling the number of action samples and the number of language rephrases both improve success rates, and compositional steering adds to gains from either form of scaling.
  • The gains transfer from simulation to a real manipulator (+17.5% over the strongest baseline), suggesting the gating logic is not an artifact of the simulator.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper's scaling law labels failure states using ground-truth actions that exist only in the offline dataset; re-deriving the law with failure defined by actual task outcomes would test whether the adaptive trigger gives the same benefit in deployment.
  • The conditional-steering recipe is generic: any large policy with internal latents and a flow or diffusion head could gate resampling or self-correction on predicted failure instead of applying it unconditionally.
  • The failure detector is trained on collected rollouts, so failures identified by the external verifier during deployment could become new labels for the detector, making the steering increasingly selective over time.
  • The conformal-threshold sweep could be replaced by learning to schedule the threshold online from reward feedback, removing the per-task calibration step.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The manuscript proposes RL2, an inference-time steering framework for vision-language-action (VLA) models. A lightweight offline RL flow-matching policy is trained on latents extracted from a frozen VLA action expert, and its velocity field is composed with the VLA's flow velocity during action generation. A failure detector (SAFE) with conformal prediction is used to trigger this compositional steering only when the base VLA is predicted to fail; otherwise, the base VLA samples are used. The paper reports a scaling-law analysis on BridgeV2 claiming that compositional steering reduces action error under high-NRMSE 'failure' tuples and degrades under low-NRMSE 'success' tuples. Based on this, it argues for adaptive steering. Experiments on SIMPLER, PolaRiS, and a real PiperX manipulator report success-rate gains over repeated and rephrased sampling baselines, with ablations on RL training, latent conditioning, and the failure-detection trigger.

Significance. If the central scaling-law asymmetry held for deployment-time failure states, the adaptive gating idea would be a meaningful contribution to test-time scaling for VLAs, since existing methods apply a uniform intervention at every timestep. The paper has clear strengths: it evaluates across three VLAs, two simulation benchmarks, and real hardware; reports timing overhead; and provides detailed hyperparameters. The real-robot gains (+17.5% over Rephrase, +14.2% over non-adaptive) are encouraging. However, the 'when to steer' insight is currently derived from an oracle NRMSE-based partition of BridgeV2 tuples, not from the task-outcome failure states that SAFE sees at deployment, and the scaling-law measurements lack error bars and quantitative fits. These gaps are load-bearing because the adaptivity is the paper's main novelty. The claimed results may be salvageable with additional analyses, but the evidence presented does not yet establish the central claim.

major comments (4)
  1. [Sec. IV and Sec. V-C] The scaling law is established on tuples partitioned by NRMSE between pi0 actions and ground-truth expert actions (top/bottom 1,024 BridgeV2 tuples), not on actual task success or failure. A high-NRMSE state need not be a state where the base policy will fail the task: demonstrations are multimodal, and a low-NRMSE action can still be followed by downstream failure. At deployment, RL2 uses SAFE, a learned binary failure predictor trained on online rollouts with task outcomes. No analysis is provided showing that SAFE-triggered states coincide with the NRMSE-defined failure set, or that the scaling-law asymmetry persists when states are labeled by true success/failure. This is load-bearing because the adaptive-vs-always comparison is the paper's headline. Concretely, I would like to see the scaling-law analysis repeated on rollout states labeled by actual task outcomes, and/or an analysis
  2. [Sec. IV, Eq. (2)] The paper claims the action-error curves follow an 'exponential power law,' but no functional form, fitted parameters, residuals, or confidence intervals are provided. It is also unclear whether the NRMSE at sample count N is the oracle-selection error (minimum over N candidates) or an average over the N candidates; this distinction determines whether a decreasing curve is meaningful at all. Please state the aggregation rule explicitly and report the fitted law with error bars. Without this, the scaling-law claim is a qualitative description rather than a quantitative result.
  3. [Sec. VI-C2, Table II] The ablation supporting the use of VLA latents compares a QAM policy conditioned on latents (39.3% for OpenVLA) with a V-GPS/CQL policy conditioned on raw observations (0.5%). These two conditions differ not only in the conditioning signal but also in the RL algorithm, network architecture, training procedure, and base VLA being steered. This is not a controlled ablation of the latent representation. A clean test would hold the RL algorithm fixed and vary only the conditioning (latents vs. raw observations), or at least add a V-GPS/CQL-on-latents condition.
  4. [Sec. VI-C1, Table III] The comparison of SAFE with CoVer as a trigger may be unfair because CoVer is a contrastive action verifier, not a failure-detection model; the paper itself shows that CoVer success/failure scores are highly correlated. The more important issue is whether the SAFE trigger realizes the Sec. IV scaling-law asymmetry. The current results show only that SAFE-gated steering improves average success over always steering. This could happen even if the scaling-law asymmetry does not hold, e.g., if SAFE selects states where the verifier happens to rank the RL-policy samples highly. I recommend reporting success-rate improvements of RL2-Adaptive over Repeated/Rephrase separately on SAFE-positive and SAFE-negative timesteps, or gating with true outcome labels, to directly test the claimed mechanism.
minor comments (5)
  1. [Eq. (2)] The notation in the NRMSE formula is ambiguous: N is said to be the number of flattened action dimensions, while T is also referenced in the denominator as 'max_T a*_i' and in the surrounding text as the set of tuples. Please define all symbols precisely.
  2. [Fig. 20] The caption states that the PolaRiS Pan Cleaning task is run with pi0 [4], but Sec. VI and Table I indicate PolaRiS experiments use pi0.5 [5]. This inconsistency should be corrected.
  3. [Sec. IV] The phrase 'exponential power law' is used repeatedly, but an exponential law and a power law are different functional forms. The authors should state the intended model explicitly.
  4. [Sec. VII] The self-stated limitations are substantial: the failure detector requires online rollout collection and per-task calibration, and the alpha-selection heuristic needs test-time evaluation. In particular, the real-world experiments had to collect new rollouts to retrain SAFE for the real environment. These points should be reflected in the abstract and conclusion, which currently describe RL2 as practically deployable without qualification.
  5. [Reproducibility] The paper says code will be released with the final paper, but no code or checkpoints are currently provided. Given the number of modules (QAM training, SAFE, CP calibration, verifier integration), releasing code and evaluation scripts is important for reproducibility of the reported gains.

Circularity Check

1 steps flagged

The headline scaling-law 'discovery' is defined on the paper's own NRMSE-based success/failure split, so the asymmetry that motivates adaptive steering is partly constructed rather than independently established.

specific steps
  1. self definitional [Sec. IV (Inference-Time Scaling Law), Eq. (2); Abstract]
    "Using π0 [4] as our base VLA, we first generate the policy actions for each tuple(s, a∗, I)within the validation set of the large-scale BridgeV2 dataset [36]. We then compute the normalized root-mean-squared error (NRMSE) between policy and ground truth actions, and extract the top and bottom 1,024 tuples based on NRMSE to define the failure and success tuple sets respectively. ... For failure tuples, RL2 compositional steering exhibits the strongest scaling behavior with significantly lower action error ... In contrast, under success tuples, most methods exhibit the opposite trend."

    The 'failure' and 'success' states are literally the top and bottom 1,024 validation tuples ranked by the NRMSE of the base VLA's actions against ground-truth actions, and the scaling law is then reported as average NRMSE on those same tuples under an oracle verifier. The label used to partition the data is the same outcome metric used to measure improvement, so the claimed asymmetry ('steering improves failure states, degrades success states') is partly guaranteed by selection on the outcome variable: high-error tuples have mechanical room for error reduction, while low-error tuples are near the floor and perturbation tends to worsen min-error. The paper then invokes this asymmetry to justify SAFE-gated adaptive steering, making the central 'when to steer' rationale depend on a self-defin

full rationale

The main circularity is confined to the framing of the scaling-law insight. The paper defines 'failure' and 'success' tuples by base-VLA NRMSE against ground truth, then 'discovers' that steering reduces NRMSE on the failure set and degrades it on the success set; that reduction is an artifact of selecting extreme values of the very metric being averaged. The subsequent claim that action diversity is most useful when the base VLA is likely to fail inherits this construction, so the adaptive-gating motivation is partially circular. The rest of the paper is substantially independent and externally grounded: the RL steering policy is trained on BridgeV2/DROID with QAM, evaluated against RoboMonkey and CoVer verifiers on SIMPLER and PolaRiS, and validated on real hardware, so those benchmark gains do not reduce to the paper's own definitions. The reliance on SAFE is not scored as load-bearing self-citation because the paper directly trains SAFE, reports ROC-AUC, and ablates against an alternative trigger; however, the real-world section acknowledges that SAFE 'could not generalize similarly' after simulation-only training, which is a robustness gap rather than a circularity. Overall, the central empirical results are real, but the headline scaling-law discovery reduces by construction, yielding a partial-circularity score of 6.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

No new physical entity or mathematical object is introduced. The framework consists of existing modules: QAM, SAFE, CoVer, and velocity composition. The invented conceptual entity is the 'adaptive scaling-law' partitioning, but it is an empirical analysis, not an entity. The main free parameters are the composition weight, CP alpha, and RL temperature, all tuned per task or per model.

free parameters (4)
  • Composition weight distribution w ~ N(0.5, 0.25) = mu=0.5, sigma=0.25
    The velocity composition weight is chosen by hand and drawn from a Gaussian with mean 0.5, variance 0.25; the paper says [19] used exhaustive sweeps but here it is set without per-task tuning (Sec. V-B).
  • Conformal prediction significance level alpha = selected per task via balanced-accuracy heuristic (top-3 among 0.05..0.50 by 0.05)
    Alpha directly controls how often steering is triggered. The paper's own heuristic selects alpha on validation rollouts to maximize balanced accuracy, so this is a fitted per-task threshold rather than a fixed a priori risk level (Sec. V-C).
  • Offline RL inverse temperature tau = 0.1 for pi0, 0.02 for pi0.5
    The temperature in QAM controls how strongly the policy is pushed toward high-Q actions. It is set per base VLA and affects how diverse the steering actions are (Appendix Table V).
  • Positive reward steps H (3) = 3
    The RL reward design uses only the first 3 steps as positive reward; this is a hand-chosen reward-shaping choice that the central steering behavior depends on (Appendix Tables V-VI).
axioms (4)
  • domain assumption VLA internal feature embeddings e_t are informative enough to train an effective RL steering policy and failure detector.
    The entire method conditions on latent features extracted from the frozen VLA action expert. If these latents lack state information required for recovering from failure, compositional steering fails. Motivated by SAFE [20], but not independently verified here.
  • domain assumption An oracle or proxy verifier can reliably rank the generated action candidates.
    The method always selects actions via a verifier (RoboMonkey or CoVer). The paper explicitly lists this as a limitation and assumes a robust verifier (Sec. VII).
  • domain assumption The flow-matching VLA velocity field can be guided by the RL policy's velocity with no representational conflict.
    Compositional steering assumes a weighted average of two velocity fields stays on valid action trajectories. This is assumed via [19], not demonstrated with ODE properties in this paper (Sec. V-B).
  • domain assumption BridgeV2/DROID ground-truth action NRMSE is a valid proxy for policy success/failure in deployment.
    Used to partition success/failure tuples for the scaling law. This is a pseudo-labeling assumption; high deviation from ground truth can include linguistically valid alternative behaviors that are actually successful (Sec. IV).

pith-pipeline@v1.3.0-daily-deepseek · 23572 in / 7210 out tokens · 50106 ms · 2026-08-01T10:18:25.995725+00:00 · methodology

0 comments
read the original abstract

Despite the impressive visuomotor capabilities enabled by Vision-Language-Action (VLA) models, their performance often degrades on challenging and out-of-domain tasks. Recent test-time steering and scaling methods improve performance without extensive data collection and retraining, but action samples often remain concentrated around similar behaviors and therefore inherit correlated failure modes. Moreover, existing methods apply the same intervention strategy at every timestep, regardless of whether the base policy is already likely to succeed. To address these limitations, we introduce $RL^2$, an adaptive inference-time steering framework that leverages Reinforcement Learning on VLA Latents. First, we train a lightweight offline RL policy conditioned on expressive latents extracted from the VLA action expert and compose its flow velocity with that of the frozen VLA during inference. This compositional steering strategy combines the behavioral priors of large-scale imitation learning with the action diversity induced by offline RL beyond dominant demonstration modes. We further discover that inference-time steering follows fundamentally different scaling laws under success and failure states, revealing that action diversity is most beneficial when the base VLA is likely to fail, but can unnecessarily perturb already-accurate actions when success is likely. Building on this insight, $RL^2$ activates compositional steering only when failure is predicted. Across the SIMPLER and PolaRiS benchmarks, $RL^2$ improves success rates by up to +17.3% in out-of-domain settings, while ablations and scaling studies demonstrate the importance of latent representations and RL training. Finally, real-world experiments demonstrate that these gains transfer beyond simulation, establishing $RL^2$ as a practical and modular steering framework for VLA deployment.

Figures

Figures reproduced from arXiv: 2607.26991 by Derek Ming Siang Tan, Guillaume Sartoretti, Qiao Gu, Shailesh Shailesh, Srikrishna Iyer, William Wei Jie Teo, Yuanliang Ju.

Figure 1
Figure 1. Figure 1: Overview: RL2 improves VLA test-time scaling by adaptively applying RL compositional steering when the base VLA is likely to fail, particularly in out-of-domain settings, without modifying the pretrained VLA. (Left) Our scaling laws indicate that RL compositional steering (green) reduces action error most effectively under failure states, but can unnecessarily perturb already-accurate success states, motiv… view at source ↗
Figure 2
Figure 2. Figure 2: Types of In-domain and OOD Tasks: While VLAs often perform well for in-domain tasks seen during training, their success rates tend to deteriorate when encountering unseen language instructions (left, 38.2% → 14.2%) [1] or unseen task environments (right, 70.2% → 36.0%) [2]. Recent methods focus on inference-time steering for VLAs to achieve better generalization to these OOD tasks without extensive data co… view at source ↗
Figure 3
Figure 3. Figure 3: Real-Robot Experiment for Tape in Toolbox Task (OOD Environment): This task is challenging for the base VLA (π0 [4]) because the tape was never seen during training. (Left) Rephrase correctly grasps the tape but often collides with the side of the toolbox. (Middle) Non-adaptive RL2 inaccurately approaches the tape, as it unnecessarily steers already-accurate VLA action samples. (Right) Adaptive RL2 relies … view at source ↗
Figure 4
Figure 4. Figure 4: Test-time Scaling Laws (Steering Improves Failure States but Degrades Success States): We observe that the action error between ground truth and predicted VLA actions (π0 [4]) consistently decreases as we scale the number of action samples across different offline RL and baseline steering approaches. Most notably, compositional steering (RL2 ) and RBF [35] exhibit the best scaling property due to their abi… view at source ↗
Figure 5
Figure 5. Figure 5: RL2 Framework (Flow-Matching VLAs): (i) For flow-matching-based VLAs like π0 [4], given language instructions and observations, RL2 first extracts action expert latents (et) from its frozen VLA action expert. (ii) The latents et is fed into the RL flow-matching steering policy, which provides guidance velocities VRL that are composed with the VLA flow velocities VV LA at each flow-matching step to generate… view at source ↗
Figure 7
Figure 7. Figure 7: SIMPLER OOD Prompt Evaluation: Using π0 [4], adaptive RL2 improves robustness under OOD language instructions [1], achieving up to +14.7% in task-wise success rate (average +10.1%) over Rephrase baseline. state vectors from the VLA action expert; we ablate different aggregation strategies (Appendix A.2) for converting these internal features into a single embedding vector et and select the best one accordi… view at source ↗
Figure 8
Figure 8. Figure 8: SIMPLER OOD Environment Evaluation: Using π0 [4], adaptive RL2 improves robustness in OOD environments [1], by up to +14.6% in task-wise success rate (average +8.5%) over the strongest Rephrase baseline. SIMPLER (OOD): We test π0 using the SIMPLER bench￾mark with the same four in-domain tasks, but with challenging red-teaming instructions [42] to simulate OOD base language prompts, via the implementation i… view at source ↗
Figure 10
Figure 10. Figure 10: CP Alpha Selection Heuristic: Our heuristic selects one of the top alphas within top-3 selection across all baselines. TABLE IV SCALING TIME ANALYSIS: RL STEERING AND FAILURE DETECTION REMAIN EFFICIENT WITH INCREASING BATCH SIZES (IN MS, ON NVIDIA RTX5090 GPU). Batch Size π0 [4] CoVer [13] QAM [37] SAFE [20] 1 232 96 12 1 2 243 100 15 1 4 265 102 16 1 8 320 107 17 2 16 434 111 21 2 32 724 114 27 2 64 1541… view at source ↗
Figure 12
Figure 12. Figure 12: PCA Heatmap Analysis: We represent VLA actions along its two principal axes via Principal Component Analysis on tuples from BridgeV2 dataset [36]. During failure, RL2 will direct action distribution (green) to be closer to the ground truth action. 1) RL2 scales well with increasing quantity and diversity of samples: From our NRMSE analysis in Sec. IV, we gain the insight that a VLA can achieve the best pe… view at source ↗
Figure 13
Figure 13. Figure 13: Results of our Real-Robot Experiments: We curate two in-domain and two challenging OOD task environments to compare RL2 against baseline approaches on-hardware. Adaptive RL2 achieves an average of +17.5% over Rephrase [13] baseline, highlighting its robustness to failure modes present in the base VLA. Similarly, adaptive RL2 achieves an average of +14.2% over non-adaptive RL2 , highlighting the importance… view at source ↗
Figure 14
Figure 14. Figure 14: Real-Robot Experiment for Screwdriver in Toolbox Task (OOD Environment): This task is challenging for the base VLA (π0 [4]) because the screwdriver was never seen during training and the scene includes a tape distractor. (Left) Rephrase correctly grasps the screwdriver but mistakenly carries the screwdriver to the tape instead of into the box. (Middle) Non-adaptive RL2 inaccurately approaches the screwdri… view at source ↗
Figure 15
Figure 15. Figure 15: Simulation Tasks: Task execution for eight SIMPLER tasks [21] (WidowX manipulator) and three PolaRiS tasks [22] (Franka manipulator) [PITH_FULL_IMAGE:figures/full_fig_p016_15.png] view at source ↗
Figure 15
Figure 15. Figure 15: Simulation Tasks: Task execution for eight SIMPLER tasks [21] (WidowX manipulator) and three PolaRiS tasks [22] (Franka manipulator) [PITH_FULL_IMAGE:figures/full_fig_p015_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: Real-World Tasks: Task execution for two in-domain and two OOD real-world task environments on the PiperX manipulator [PITH_FULL_IMAGE:figures/full_fig_p017_16.png] view at source ↗
Figure 16
Figure 16. Figure 16: Real-World Tasks: Task execution for two in-domain and two OOD real-world task environments on the PiperX manipulator [PITH_FULL_IMAGE:figures/full_fig_p016_16.png] view at source ↗
Figure 17
Figure 17. Figure 17: CP Alpha Selection Heuristics: Examples of our Conformal Prediction α selection heuristic comparing balanced accuracy against significance level α plots for real-world tasks, with sample task-wise CP plots (left) and the combined CP plot for OOD tasks (right). From these plots, we can select the top-k alpha (3, in practice) to evaluate and determine the best CP band to be selected for each task. C. Additi… view at source ↗
Figure 17
Figure 17. Figure 17: CP Alpha Selection Heuristics: Examples of our Conformal Prediction α selection heuristic comparing balanced accuracy against significance level α plots for real-world tasks, with sample task-wise CP plots (left) and the combined CP plot for OOD tasks (right). From these plots, we can select the top-k alpha (3, in practice) to evaluate and determine the best CP band to be selected for each task. C. Additi… view at source ↗
Figure 18
Figure 18. Figure 18: Comparison of SAFE Failure Probabilities and CoVer Scores: For both π0 and π0.5 as the base VLA, we visualize SAFE [20] failure probabilities (left) and CoVer [13] VLM normalized score (right) against normalized episode progress. The solid lines and their corresponding colored bands represent the mean and standard deviation across a large number of episodes n. SAFE acts as an effective intervention trigge… view at source ↗
Figure 18
Figure 18. Figure 18: Comparison of SAFE Failure Probabilities and CoVer Scores: For both π0 and π0.5 as the base VLA, we visualize SAFE [20] failure probabilities (left) and CoVer [13] VLM normalized score (right) against normalized episode progress. The solid lines and their corresponding colored bands represent the mean and standard deviation across a large number of episodes n. SAFE acts as an effective intervention trigge… view at source ↗
Figure 19
Figure 19. Figure 19: SIMPLER Experiment for Spoon on Towel Task (OOD Prompt): This task is challenging for the base VLA (π0 [4]) because the base language prompt was never seen during training. (Left) Rephrase repeatedly fails to grasp the spoon during replanning. (Right) Adaptive RL2 is more likely to successfully grasp the spoon during replanning, after failure is preemptively detected via Conformal Prediction (CP) [20] [P… view at source ↗
Figure 19
Figure 19. Figure 19: SIMPLER Experiment for Spoon on Towel Task (OOD Prompt): This task is challenging for the base VLA (π0 [4]) because the base language prompt was never seen during training. (Left) Rephrase repeatedly fails to grasp the spoon during replanning. (Right) Adaptive RL2 is more likely to successfully grasp the spoon during replanning, after failure is preemptively detected via Conformal Prediction (CP) [20] [P… view at source ↗
Figure 20
Figure 20. Figure 20: PolaRiS Experiment for Pan Cleaning Task (OOD Prompt): This task is challenging for the base VLA (π0 [4]) because the base language prompt was never seen during training. (Left) Rephrase repeatedly fails to grasp the sponge during replanning and eventually gets distracted by the coke can. (Right) Adaptive RL2 is more likely to successfully grasp the sponge during replanning, after failure is preemptively … view at source ↗
Figure 20
Figure 20. Figure 20: PolaRiS Experiment for Pan Cleaning Task (OOD Prompt): This task is challenging for the base VLA (π0 [4]) because the base language prompt was never seen during training. (Left) Rephrase repeatedly fails to grasp the sponge during replanning and eventually gets distracted by the coke can. (Right) Adaptive RL2 is more likely to successfully grasp the sponge during replanning, after failure is preemptively … view at source ↗
Figure 21
Figure 21. Figure 21: SIMPLER Experiment for Toy Dinosaur on Towel Task (OOD Environment): This task is challenging for the base VLA (π0 [4]) because the toy dinosaur and elephant were never seen during training. (Left) Rephrase initially approaches the dinosaur but is eventually distracted by the elephant. (Middle) Non-adaptive RL2 consistently approaches the dinosaur but inaccurately attempts to grasp it. (Right) Adaptive RL… view at source ↗
Figure 21
Figure 21. Figure 21: SIMPLER Experiment for Toy Dinosaur on Towel Task (OOD Environment): This task is challenging for the base VLA (π0 [4]) because the toy dinosaur and elephant were never seen during training. (Left) Rephrase initially approaches the dinosaur but is eventually distracted by the elephant. (Middle) Non-adaptive RL2 consistently approaches the dinosaur but inaccurately attempts to grasp it. (Right) Adaptive RL… view at source ↗
Figure 22
Figure 22. Figure 22: Real-Robot Experiment for Carrot on Plate Task (OOD Prompt): This task is challenging for the base VLA (π0 [4]) because the base language prompt was never seen during training. (Left) Rephrase repeatedly fails to grasp the carrot during replanning. (Right) Adaptive RL2 is more likely to successfully grasp the carrot during replanning, after failure is preemptively detected via Conformal Prediction (CP) [2… view at source ↗
Figure 22
Figure 22. Figure 22: Real-Robot Experiment for Carrot on Plate Task (OOD Prompt): This task is challenging for the base VLA (π0 [4]) because the base language prompt was never seen during training. (Left) Rephrase repeatedly fails to grasp the carrot during replanning. (Right) Adaptive RL2 is more likely to successfully grasp the carrot during replanning, after failure is preemptively detected via Conformal Prediction (CP) [2… view at source ↗
Figure 23
Figure 23. Figure 23: Real-Robot Experiment for Cube in Toolbox Task (OOD Prompt): This task is challenging for the base VLA (π0 [4]) because the base language prompt was never seen during training. (Left) Rephrase repeatedly fails to grasp the cube during replanning. (Right) Adaptive RL2 is more likely to successfully grasp the cube during replanning, after failure is preemptively detected via Conformal Prediction (CP) [20]. … view at source ↗
Figure 23
Figure 23. Figure 23: Real-Robot Experiment for Cube in Toolbox Task (OOD Prompt): This task is challenging for the base VLA (π0 [4]) because the base language prompt was never seen during training. (Left) Rephrase repeatedly fails to grasp the cube during replanning. (Right) Adaptive RL2 is more likely to successfully grasp the cube during replanning, after failure is preemptively detected via Conformal Prediction (CP) [20]. … view at source ↗

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

145 extracted references · 11 linked inside Pith

  1. [1]

    From intention to execution: Probing the generalization boundaries of vision-language-action mod- els,

    I. Fang, J. Zhang, S. Tong, and C. Feng, “From intention to execution: Probing the generalization boundaries of vision-language-action mod- els,”arXiv preprint arXiv2506.09930, 2025

    Pith/arXiv arXiv 2025

  2. [2]

    Interleave-VLA: Enhancing robot ma- nipulation with image-text interleaved instructions,

    C. Fan, X. Jia, Y . Sun, Y . Wang, J. Wei, Z. Gong, X. Zhao, M. Tomizuka, X. Yang, J. Yan, and M. Ding, “Interleave-VLA: Enhancing robot ma- nipulation with image-text interleaved instructions,” inThe Fourteenth International Conference on Learning Representations, 2026

    2026

  3. [3]

    Openvla: An open-source vision-language-action model,

    M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. P. Foster, P. R. Sanketiet al., “Openvla: An open-source vision-language-action model,” inProceedings of The 8th Conference on Robot Learning, 2025

    2025

  4. [4]

    π 0: A vision-language-action flow model for general robot control,

    K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, K. Hausmanet al., “π 0: A vision-language-action flow model for general robot control,”arXiv preprint arXiv:2410.24164, 2026

    Pith/arXiv arXiv 2026

  5. [5]

    π 0.5: a vision- language-action model with open-world generalization,

    K. Black, N. Brown, J. Darpinian, K. Dhabalia, D. Driess, A. Esmail, M. R. Equi, C. Finn, N. Fusai, M. Y . Gallikeret al., “π 0.5: a vision- language-action model with open-world generalization,” inProceedings of The 9th Conference on Robot Learning, 2025

    2025

  6. [6]

    Molmoact: Action reasoning models that can reason in space,

    J. Lee, J. Duan, H. Fang, Y . Deng, S. Liu, B. Li, B. Fang, J. Zhang, Y . R. Wang, S. Leeet al., “Molmoact: Action reasoning models that can reason in space,”arXiv preprint arXiv:2508.07917, 2025

    Pith/arXiv arXiv 2025

  7. [7]

    Open x-embodiment: Robotic learning datasets and rt-x models

    A. O’Neill, A. Rehman, A. Maddukuri, A. Gupta, A. Padalkar, A. Lee, A. Pooley, A. Gupta, A. Mandlekar, A. Jainet al., “Open x-embodiment: Robotic learning datasets and rt-x models.” in2024 IEEE International Conference on Robotics and Automation, 2024

    2024

  8. [8]

    GRAPE: Generalizing robot policy via preference alignment,

    Z. Zhang, K. Zheng, Z. Chen, J. Jang, Y . Li, S. Han, C. Wang, M. Ding, D. Fox, and H. Yao, “GRAPE: Generalizing robot policy via preference alignment,” inarXiv preprint arXiv:2411.19309, 2025

    Pith/arXiv arXiv 2025

  9. [9]

    Robotic control via embodied chain-of-thought reasoning,

    M. Zawalski, W. Chen, K. Pertsch, O. Mees, C. Finn, and S. Levine, “Robotic control via embodied chain-of-thought reasoning,” inProceed- ings of The 8th Conference on Robot Learning, 2024

    2024

  10. [10]

    Fast ecot: Efficient embodied chain-of-thought via thoughts reuse,

    Z. Duan, Y . Zhang, S. Geng, G. Liu, J. Boedecker, and C. X. Lu, “Fast ecot: Efficient embodied chain-of-thought via thoughts reuse,”IEEE International Conference on Robotics and Automation, 2025

    2025

  11. [11]

    Steering your generalists: Improving robotic foundation models via value guidance,

    M. Nakamoto, O. Mees, A. Kumar, and S. Levine, “Steering your generalists: Improving robotic foundation models via value guidance,” Proceedings of The 8th Conference on Robot Learning, 2024

    2024

  12. [12]

    Robomonkey: Scaling test-time sampling and verifi- cation for vision-language-action models,

    J. Kwok, C. Agia, R. Sinha, M. Foutter, S. Li, I. Stoica, A. Mirhoseini, and M. Pavone, “Robomonkey: Scaling test-time sampling and verifi- cation for vision-language-action models,” inProceedings of The 9th Conference on Robot Learning, 2025

    2025

  13. [13]

    Scaling verification can be more effective than scaling policy learning for vision-language-action alignment,

    J. Kwok, X. Zhang, M. Xu, Y . Liu, A. Mirhoseini, C. Finn, and M. Pavone, “Scaling verification can be more effective than scaling policy learning for vision-language-action alignment,” inEuropean Conference on Computer Vision, 2026

    2026

  14. [14]

    From foresight to fore- thought: Vlm-in-the-loop policy steering via latent alignment,

    Y . Wu, R. Tian, G. Swamy, and A. Bajcsy, “From foresight to fore- thought: Vlm-in-the-loop policy steering via latent alignment,”Robotics: Science and Systems (RSS), 2025

    2025

  15. [15]

    Do what you say: Steering vision-language-action models via runtime reasoning-action alignment verification,

    Y . Wu, A. Li, T. Hermans, F. Ramos, A. Bajcsy, and C. P´erez-D’Arpino, “Do what you say: Steering vision-language-action models via runtime reasoning-action alignment verification,”2026 IEEE International Con- ference on Robotics & Automation, 2026

    2026

  16. [16]

    Dynaguide: Steering diffusion policies with active dynamic guidance,

    M. Du and S. Song, “Dynaguide: Steering diffusion policies with active dynamic guidance,” inThe Thirty-ninth Conference on Neural Information Processing Systems, 2026

    2026

  17. [17]

    VLS: Steering pretrained robot policies via vision-language models,

    S. Liu, I. S. Singh, Y . Xu, J. Duan, and R. Krishna, “VLS: Steering pretrained robot policies via vision-language models,”arXiv preprint arXiv:2602.03973, 2026

    arXiv 2026

  18. [18]

    Towards deploying vla without fine-tuning: Plug-and-play inference- time vla policy steering via embodied evolutionary diffusion,

    Z. Li, J. Liu, Z. Dong, T. Teng, Q. Rouxel, D. Caldwell, and F. Chen, “Towards deploying vla without fine-tuning: Plug-and-play inference- time vla policy steering via embodied evolutionary diffusion,”arXiv preprint arXiv:2511.14178, 2026

    Pith/arXiv arXiv 2026

  19. [19]

    Compose your policies! improving diffusion-based or flow-based robot policies via test-time distribution- level composition,

    J. Cao, Y . Huang, H. Guo, Q. Zhang, R. Zhang, W. Mai, M. Nan, J. Wang, H. Cheng, J. Sunet al., “Compose your policies! improving diffusion-based or flow-based robot policies via test-time distribution- level composition,” inThe Fourteenth International Conference on Learning Representations, 2026

    2026

  20. [20]

    Safe: Multitask failure detection for vision-language-action models,

    Q. Gu, Y . Ju, S. Sun, I. Gilitschenski, H. Nishimura, M. Itkina, and F. Shkurti, “Safe: Multitask failure detection for vision-language-action models,”Advances in Neural Information Processing Systems, 2026

    2026

  21. [21]

    Evaluating real-world robot manipulation policies in simulation,

    X. Li, K. Hsu, J. Gu, K. Pertsch, O. Mees, H. R. Walke, C. Fu, I. Lunawat, I. Siehet al., “Evaluating real-world robot manipulation policies in simulation,”arXiv preprint arXiv:2405.05941, 2024

    Pith/arXiv arXiv 2024

  22. [22]

    Polaris: Scalable real-to-sim evaluations for generalist robot policies,

    A. Jain, M. Zhang, K. Arora, W. Chen, M. Torne, M. Z. Irshad, S. Zakharov, Y . Wanget al., “Polaris: Scalable real-to-sim evaluations for generalist robot policies,”arXiv preprint arXiv:2512.16881, 2025

    arXiv 2025

  23. [23]

    π ∗ 0.6: a vla that learns from experience,

    P. Intelligence, A. Amin, R. Aniceto, A. Balakrishna, K. Black, K. Con- ley, G. Connors, J. Darpinian, K. Dhabalia, J. DiCarlo, D. Driess et al., “π ∗ 0.6: a vla that learns from experience,”arXiv preprint arXiv:2511.14759, 2026

    Pith/arXiv arXiv 2026

  24. [24]

    SimpleVLA-RL: Scaling VLA training via reinforcement learning,

    H. Li, Y . Zuo, J. Yu, Y . Zhang, Y . Zhaohui, K. Zhang, X. Zhu, Y . Zhang, T. Chen, G. Cuiet al., “SimpleVLA-RL: Scaling VLA training via reinforcement learning,” inThe Fourteenth International Conference on Learning Representations, 2026

    2026

  25. [25]

    Conrft: A reinforced fine-tuning method for vla models via consistency policy,

    Y . Chen, S. Tian, S. Liu, Y . Zhou, H. Li, and D. Zhao, “Conrft: A reinforced fine-tuning method for vla models via consistency policy,” in Proceedings of Robotics: Science and Systems, RSS, 2025

    2025

  26. [26]

    Policy decorator: Model-agnostic online refinement for large policy model,

    X. Yuan, T. Mu, S. Tao, Y . Fang, M. Zhang, and H. Su, “Policy decorator: Model-agnostic online refinement for large policy model,” in Thirteenth International Conference on Learning Representations, 2025

    2025

  27. [27]

    Self-improving vision-language-action models with data generation via residual RL,

    W. Xiao, H. Lin, A. Peng, H. Xue, T. He, Z. Luo, Y . Xie, F. Hu, L. Fan, G. Shi, and Y . Zhu, “Self-improving vision-language-action models with data generation via residual RL,” inThe Fourteenth International Conference on Learning Representations, 2026

    2026

  28. [28]

    Steering your diffusion policy with latent space reinforcement learning,

    A. Wagenmaker, M. Nakamoto, Y . Zhang, S. Park, W. Yagoub, A. Naga- bandi, A. Gupta, and S. Levine, “Steering your diffusion policy with latent space reinforcement learning,”Proceedings of The 9th Conference on Robot Learning, 2025

    2025

  29. [29]

    Rl token: Bootstrapping online rl with vision-language-action models,

    C. Xu, J. T. Springenberg, M. Equi, A. Amin, A. Esmail, S. Levine, and L. Ke, “Rl token: Bootstrapping online rl with vision-language-action models,”arXiv preprint arXiv:2604.23073, 2026

    Pith/arXiv arXiv 2026

  30. [30]

    OnetwoVLA: A unified vision-language-action model with adaptive reasoning,

    F. Lin, R. Nai, Y . Hu, J. You, J. Zhao, and Y . Gao, “OnetwoVLA: A unified vision-language-action model with adaptive reasoning,” inThe Fourteenth International Conference on Learning Representations, 2026

    2026

  31. [31]

    Recurrent-depth vla: Implicit test-time compute scaling of vision-language-action models via latent iterative reasoning,

    Y . Tur, J. Naghiyev, H. Fang, W.-C. Tsai, J. Duan, D. Fox, and R. Krishna, “Recurrent-depth vla: Implicit test-time compute scaling of vision-language-action models via latent iterative reasoning,”arXiv preprint arXiv:2602.07845, 2026

    arXiv 2026

  32. [32]

    Vla- attc: Adaptive test-time compute for vla models with relative action critic model,

    W. Li, X. Su, Y . Cao, H. Xu, X. Xia, S. You, Y . Chen, and C. Xu, “Vla- attc: Adaptive test-time compute for vla models with relative action critic model,”arXiv preprint arXiv2605.01194, 2026

    Pith/arXiv arXiv 2026

  33. [33]

    Scale: Self-uncertainty conditioned adaptive looking and execution for vision- language-action models,

    H. Choi, D. Ahn, Y . Lee, T. Kang, S. Cho, and J. Choi, “Scale: Self-uncertainty conditioned adaptive looking and execution for vision- language-action models,” inThe Fourteenth International Conference on Learning Representations, 2026

    2026

  34. [34]

    Diffusion models beat GANs on image synthesis,

    P. Dhariwal and A. Q. Nichol, “Diffusion models beat GANs on image synthesis,” inThe Thirty-fifth Conference on Neural Information Processing Systems, 2021

    2021

  35. [35]

    Tree-guided diffusion planner,

    H. Jeon, C. Min, and J. Park, “Tree-guided diffusion planner,”Advances in Neural Information Processing Systems, 2026

    2026

  36. [36]

    Bridgedata v2: A dataset for robot learning at scale,

    H. Walke, K. Black, A. Lee, M. J. Kim, M. Du, C. Zheng, T. Zhao, P. Hansen-Estruch, Q. Vuong, Heet al., “Bridgedata v2: A dataset for robot learning at scale,” inThe 7th Conference on Robot Learning, 2023

    2023

  37. [37]

    Q-learning with adjoint matching,

    Q. Li and S. Levine, “Q-learning with adjoint matching,” inThe Fourteenth International Conference on Learning Representations, 2026

    2026

  38. [38]

    Droid: A large-scale in-the-wild robot manipulation dataset,

    A. Khazatsky, K. Pertsch, S. Nair, A. Balakrishna, S. Dasari, S. Karam- cheti, S. Nasiriany, M. K. Srirama, and L. Y . C. et al., “Droid: A large-scale in-the-wild robot manipulation dataset,”Robotics: Science and Systems (RSS), 2024

    2024

  39. [39]

    Flow q-learning,

    S. Park, Q. Li, and S. Levine, “Flow q-learning,” inForty-second International Conference on Machine Learning, 2025

    2025

  40. [40]

    Conservative q-learning for offline reinforcement learning,

    A. Kumar, A. Zhou, G. Tucker, and S. Levine, “Conservative q-learning for offline reinforcement learning,” inAdvances in Neural Information Processing Systems, 2020

    2020

  41. [41]

    Can we detect failures without failure data? uncertainty-aware runtime failure detection for imitation learning policies,

    C. Xu, T. K. Nguyen, E. Dixon, C. Rodriguez, P. Miller, R. Lee, P. Shah, R. Ambrus, H. Nishimura, and M. Itkina, “Can we detect failures without failure data? uncertainty-aware runtime failure detection for imitation learning policies,”arXiv preprint arXiv2503.08558, 2024

    Pith/arXiv arXiv 2024

  42. [42]

    Embodied red teaming for auditing robotic foundation models,

    S. Karnik, Z.-W. Hong, N. Abhangi, Y .-C. Lin, T.-H. Wang, and P. Agrawal, “Embodied red teaming for auditing robotic foundation models,”arXiv preprint arXiv2411.18676, 2024

    Pith/arXiv arXiv 2024

  43. [43]

    OGBench: Bench- marking offline goal-conditioned RL,

    S. Park, K. Frans, B. Eysenbach, and S. Levine, “OGBench: Bench- marking offline goal-conditioned RL,” inThe Thirteenth International Conference on Learning Representations, 2025. 13 APPENDIX A. Additional Implementation Details In this section, we provide additional training details regard- ing our RL steering policies and failure detection models. 1)RL ...

    2025

  44. [51]

    Continued on next page 19 TABLE IX LANGUAGEINSTRUCTIONSREPHRASES(CONTINUED FROM PREVIOUS PAGE)

    Place the vegetable on the ceramic plate. Continued on next page 19 TABLE IX LANGUAGEINSTRUCTIONSREPHRASES(CONTINUED FROM PREVIOUS PAGE). Task Name Original Instruction Red-Team Instruction Generated Rephrases Spoon on Towel (SIMPLER) Put the spoon on the towel. Set the spoon exactly in the middle of the towel

  45. [52]

    Place the spoon in the center of the towel

  46. [58]

    Lay the green-handled spoon in the middle of the fabric

  47. [59]

    Stack Cubes (SIMPLER) Stack the green block on the yellow block

    Set the spoon in the center of the towel. Stack Cubes (SIMPLER) Stack the green block on the yellow block. Arrange the lush green element atop the yellowish-orange element

  48. [60]

    Place the green cube on the yellow block

  49. [61]

    Put the grassy block on top of the yellow piece

  50. [62]

    Set the green block over the yellow cube

  51. [63]

    Stack the green piece onto the yellowish block

  52. [64]

    Position the lush cube above the yellow element

  53. [65]

    Place the green cube on the yellow cube

  54. [66]

    Set the green block on top of the yellow block

  55. [67]

    Eggplant in Basket (SIMPLER) Put eggplant into yellow basket

    Put the green element over the yellow cube. Eggplant in Basket (SIMPLER) Put eggplant into yellow basket. Arrange the eggplant neatly in the yellow bin

  56. [68]

    Place the vegetable in the yellow rack

  57. [69]

    Put the eggplant in the yellow holder

  58. [70]

    Set the purple vegetable into the yellow bin

  59. [71]

    Organize the eggplant in the yellow rack

  60. [72]

    Position the vegetable in the yellow holder

  61. [73]

    Place the purple vegetable in the yellow container

  62. [74]

    Set the eggplant into the dish rack

  63. [75]

    Orange Juice on Plate (SIMPLER) Put orange juice on plate

    Move the purple produce to the yellow bin. Orange Juice on Plate (SIMPLER) Put orange juice on plate. – 1. Set the carton of juice onto the dish

  64. [76]

    Move the orange drink to the saucer

  65. [77]

    Lay the juice carton on the platter

  66. [78]

    Position the orange juice on the dish

  67. [79]

    Put the drink carton on the yellow plate

  68. [80]

    Place the orange drink container onto the platter

  69. [81]

    Set the juice on the yellow saucer

  70. [82]

    Spoon on Towel (Google) (SIMPLER) Put the spoon on the towel google

    Move the orange box to the plate. Spoon on Towel (Google) (SIMPLER) Put the spoon on the towel google. – 1. Place the spoon in the center of the towel

  71. [83]

    Put the utensil on the middle of the cloth

  72. [84]

    Position the scoop at the center of the fabric

  73. [85]

    Lay the green-handled spoon in the middle of the blue towel

  74. [86]

    Set the utensil in the central part of the towel

  75. [87]

    Position the spoon on the center of the blue towel

  76. [88]

    Set the spoon exactly in the middle of the towel

  77. [89]

    Toy Dinosaur on Towel (SIMPLER) Put the toy dinosaur on the towel

    Set the spoon in the center of the towel. Toy Dinosaur on Towel (SIMPLER) Put the toy dinosaur on the towel. – 1. Place the toy on the cloth

  78. [90]

    Move the figure to the blue fabric

  79. [91]

    Transfer the creature to the piece of cloth

  80. [92]

    Lay the toy on the rag

Showing first 80 references.