REVIEW 5 major objections 5 minor 8 cited by
A single offline training objective can make vision-language-action policies robust to multi-modal noise by adversarially training against both worst-case action perturbations and semantically-preserving input corruptions selected by a band
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-04 14:52 UTC pith:Z7JAYGZV
load-bearing objection RobustVLA is a genuinely useful multi-modal robustness study for VLAs and the training method works on the tested 17 perturbations, but the lack of any held-out perturbation family leaves the generalization claim unproven. the 5 major comments →
RobustVLA: On Robustness of Vision-Language-Action Model against Multi-Modal Perturbations
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central claim is that a VLA policy can be made substantially more robust to multi-modal noise by a single offline training objective that treats both the action output and the input observation as attack surfaces. The output branch defines worst-case action noise as the bounded PGD maximizer of the flow-matching loss—the difference between the predicted velocity field and the rectified flow target—and trains the action head to match the clean and adversarially perturbed flow distributions. The input branch enforces consistent actions across perturbations that preserve task semantics, with an upper confidence bound (UCB) selector automatically picking the currently most harmful corruption
What carries the argument
The central machinery is the flow-matching objective used as a proxy for action quality: conditional flow matching defines a velocity field pushing Gaussian noise into the action distribution, and the squared mismatch between predicted velocity and the rectified-flow target is treated as the loss that adversarial action noise maximizes. On the output side, projected gradient descent computes an ℓ_p-bounded worst-case action perturbation; on the input side, a UCB bandit selects the most harmful semantically-preserving corruption, and both are trained against jointly with the clean flow-matching loss. This combined objective smooths the action head, penalizes outliers, and enforces local input
Load-bearing premise
The load-bearing premise is that maximizing the flow-matching loss finds genuinely harmful action noise—i.e., that flow loss is monotonically related to task failure—and that all 17 input perturbations preserve task semantics, so enforcing action invariance under them is the right regularizer.
What would settle it
Take a trained RobustVLA and evaluate on an action perturbation uncorrelated with the training ones, such as multiplicative scaling bursts instead of additive noise. If success rate collapses while the flow-matching loss stays low, or if a high-flow-loss action still succeeds, the proxy assumption fails. A simpler test: measure whether success rate under PGD action attack tracks flow loss across noise levels on LIBERO.
If this is right
- Robustness gains span all four tested modalities on both diffusion-based and autoregressive backbones, suggesting that modality-agnostic training is more effective than visual-only defenses.
- Clean performance is nearly preserved, so the robustness improvement does not come at the expense of in-distribution task success.
- Inference stays cheap because only training changes—no external large model, segmentation, or runtime intervention is needed at test time.
- Action-noise robustness also helps under input noise and vice versa, implying cross-modal transfer of robustness within the same training objective.
- Real-world low-data deployments benefit disproportionately, with large success-rate gains when only 25 demonstrations are available.
Where Pith is reading between the lines
- One extension not pursued in the paper: the same worst-case action-noise training recipe could be applied to other diffusion- or flow-based robot policies beyond pi0, and its transfer could be measured on a different benchmark to test whether the mechanism is policy-agnostic.
- The UCB scheduling mechanism may be useful more broadly as a way to balance multiple augmentations in imitation learning; this could be tested on tasks where the set of corruptions is much larger than 17.
- The 17-perturbation evaluation suite could serve as a portable robustness checklist for VLA releases, allowing the community to track whether future architectures are improving general robustness or merely fitting the known corruptions.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies robustness of vision-language-action (VLA) models to perturbations across four modalities: action, observation, environment, and instruction. It first evaluates mainstream VLAs (π0, OpenVLA, π0-FAST, BYOVLA) under 17 perturbation types on LIBERO, reporting that action perturbations are the most damaging, that a visual-robust method (BYOVLA) does not transfer to other modalities, and that π0 is the most robust backbone. It then proposes RobustVLA, which combines (i) output robustness through adversarial training against worst-case action noise defined by the maximum of the flow-matching loss, and (ii) input robustness via consistency regularization under semantically preserving input perturbations, with a UCB-based bandit to select the most harmful perturbation during training. Experiments on LIBERO report absolute gains of 12.6% over BYOVLA on π0 and 10.4% on OpenVLA across the 17 perturbations, plus real-world FR5 experiments reporting a 65.6% gain over baselines under four modalities of physical noise with 25 demonstrations. The abstract also claims a 30% gain with abundant demonstrations, though no such experiment is reported.
Significance. If the results hold, this is a valuable contribution to VLA robustness: it broadens the evaluation from purely visual disturbances to a multi-modal set, proposes a training method that works on both diffusion-based and autoregressive VLAs, and includes real-world validation with physically implemented noise. The paper is also transparent in providing code, a detailed appendix, and ablations that separate input and output robustness. The main results are plausible but are currently weakened by training/evaluation overlap, an unvalidated proxy for action quality, and missing statistical detail. These issues are addressable and do not, at this stage, invalidate the contribution, but they must be fixed before the central claim of general multi-modal robustness can be accepted.
major comments (5)
- [Section 4.1, Eq. (3)] The output-robustness branch is built on the assumption that the flow-matching loss is an empirical measure of action quality: 'we use the loss of the flow matching objective ... as an empirical measure of action quality.' This is load-bearing because the worst-case action noise δ is defined as the PGD maximizer of this proxy. The paper does not validate the proxy. A concrete test would be to compare task success under the PGD action noise (same ℓ∞ bound) against random noise of equal magnitude on the clean π0 model. If PGD noise does not degrade success more than random noise, the 'worst-case' adversary is not the right one, and the reported robustness gains may reflect fitting to the evaluation perturbations rather than a principled mechanism. Please add this validation or, alternatively, justify the proxy with empirical correlation data.
- [Section 4.2 vs. Section 5.1, Table 1] Training and evaluation reuse the same 17 perturbation families. Section 4.2 trains the input-robustness branch with UCB selecting among exactly the perturbation types evaluated in Table 1. For action noise, the test-time Action Bias σ=0.03 is equal to the training-time ℓ∞ bound ϵ=0.03 (Appendix B.1.3), so the evaluation is partially in-distribution. The real-world experiments (Section 5.3) say observation noise uses 'the same perturbations as in simulation' and the other physical noise sources are still within the trained categories. Without a held-out perturbation family (e.g., a new observation corruption, a different noise distribution, or an unseen combination not in Fig. 2), the reported gains are consistent with benchmark-fitting rather than general multi-modal robustness. Please add at least one held-out perturbation family and show that RobustVLA retains a meaningful advantage.
- [Abstract and Section 5.3] The abstract claims 'Even with abundant demos, our method still outperform pi0 by 30% success rate,' but no abundant-demo experiment appears in Section 5.3 or in the appendix. The real-world section reports only 25 demonstration trajectories. This is an unsupported part of the central real-world claim. Either add the experiment with a larger demonstration count (e.g., 100+ demos) or remove the claim from the abstract.
- [Table 1] Table 1 reports average success rates without error bars, multiple seeds, or statistical significance. LIBERO evaluation is stochastic, and the perturbation implementations are random (e.g., random flips, dead pixel masks, action noise draws). The claimed 12.6% average absolute gain over BYOVLA could be within run-to-run variance. Statistical significance is reported only for the mixed-perturbation experiment (Section 5.2). Please provide means and standard deviations over at least 3 independent seeds (or, if compute constraints are severe, over 3 different random perturbation draws) and a paired significance test across the 17 conditions for the main table.
- [Section 4.2, Eq. (6)-(8)] The UCB-based perturbation selection is presented as a key component of the input-robustness branch, but there is no ablation comparing UCB selection to a simpler uniform random sampling over the 17 perturbation types or to fixed weights. Without such an ablation, it is unclear whether the reported gains come from the bandit mechanism or simply from training on all perturbations. Please add an ablation 'RobustVLA with uniform perturbation sampling' (same λ_in, same loss) to isolate the contribution of the UCB selector.
minor comments (5)
- [Section 4.1, text after Eq. (3)] There is a typo in the definition of the perturbed action: 'ˆAτ t = τ ˆAτ t + (1−τ)A0 t' should be 'ˆAτ t = τ ˆA1 t + (1−τ)A0 t'.
- [Section 3.2, paragraph 1] The sentence 'For example, the success rate of π0 are reduced to 52.4%' has a subject-verb agreement error ('are' should be 'is').
- [Section 6] Typo: 'evalute' should be 'evaluate'.
- [Appendix A.4] The Adversarial Prompts example includes 'bBBowl' (stray capital letters). This appears intentional as an OCR typo, but it may confuse readers; consider clarifying that this is part of the perturbation.
- [Figure 3 and Table 1] The figure captions and table headers call some perturbations 'Gaussian Noise' in both action and observation modalities. Since both appear in the same table, consider renaming them 'Action Gaussian' and 'Visual Gaussian' (as used in Fig. 3b) to avoid ambiguity.
Circularity Check
Training and evaluation use the same 17 perturbation families, so the headline robustness gains are in-distribution; no held-out perturbations establish general multi-modal robustness.
specific steps
-
fitted input called prediction
[Section 4.2 (Eqs. 6-8); Section 5.1 (Table 1)]
"At each training step n, the algorithm selects an uncertainty ωi ∈ Ω to train the objective in Eq. 6. ... Consistent with the robustness evaluation in Section 3.1, we assess all methods under the same 17 perturbations using the LIBERO benchmark and its recommended setup."
The robust input objective (Eq. 8) is minimized under perturbations ωi* chosen by UCB from the same set Ω that defines the 17 evaluation perturbations in Section 3.1/Fig. 2. Therefore the reported average gains (12.6% on π0, 10.4% on OpenVLA) are measured on the exact perturbation families used for training. The 'prediction' that RobustVLA is robust to these 17 perturbations is not an independent generalization result; it is the direct outcome of minimizing flow-matching loss under those perturbation operators. No perturbation family is held out, so the central claim cannot distinguish general multi-modal robustness from fitting to the 17-type taxonomy.
-
other
[Appendix B.1.3 (Table 7); Appendix A.1 (Action Bias)]
"adv_epsilon 0.03 ... In the evaluation, the value of σ was set to 0.03."
The output-robustness branch trains against PGD action perturbations within an ℓ∞ ball of radius 0.03 (adv_epsilon, Table 7). The Action Bias evaluation uses a constant action offset of exactly σ = 0.03 (Appendix A.1), i.e., a perturbation on the boundary of the training adversary's ball. The observed Action Bias gain (23.0→42.3 on π0) is therefore trained at the test magnitude, not a held-out extrapolation. This is a second instance where the reported robustness is in-distribution with the training objective.
full rationale
The primary derivation chain is circular in the benchmark sense: Section 4.2 trains RobustVLA by minimizing the flow-matching loss under perturbations selected by UCB from the same Ω that defines the 17 evaluation perturbations in Section 5. The headline robustness gains are thus in-distribution measurements of the training augmentations. The action-noise branch is similar: the PGD bound ε=0.03 equals the Action Bias test σ=0.03. Some independent content exists: clean performance is roughly preserved (95.5 vs 96.0), the method also helps OpenVLA, and the real-world FR5 experiments physically re-instantiate noise categories (speech recognition, motor calibration, lighting). However, the real-world observation noise is 'the same perturbations as in simulation', the sample size is small (10 trials/task), and the abstract's 'Even with abundant demos... 30%' claim is unsupported by any reported experiment in Section 5.3 or the appendix. There is no load-bearing self-citation chain or imported uniqueness theorem. Score 6 reflects that the central benchmark claim reduces largely—but not entirely—to training on the evaluation perturbation set.
Axiom & Free-Parameter Ledger
free parameters (7)
- lambda_in, lambda_out (loss weights) =
1.0
- action PGD epsilon delta =
0.03
- observation PGD epsilon eta =
8/255
- PGD steps and alpha (action/obs) =
3 steps, alpha 0.01 and 2/255
- UCB exploration coefficient alpha =
1.0
- UCB window size / EMA decay / min samples =
100 / 0.9 / 10
- Evaluation noise levels per perturbation =
e.g., action uniform sigma=0.04, action Gaussian sigma=0.3, visual Gaussian sigma=70, rotation +/-20 deg, shift 0.15
axioms (3)
- domain assumption The 17 input perturbations are semantics-preserving: the optimal action is unchanged under perturbed observations/instructions/environment.
- ad hoc to paper The flow-matching loss magnitude is a valid proxy for action quality / task success.
- domain assumption Training against small PGD-bounded perturbations transfers to the larger, random test-time noise distributions.
Cite this review
Pith. "Pith review of RobustVLA: On Robustness of Vision-Language-Action Model against Multi-Modal Perturbations." pith.science (2026). https://pith.science/paper/Z7JAYGZV
@misc{pith2026251000037,
author = {Pith},
title = {Pith review of: RobustVLA: On Robustness of Vision-Language-Action Model against Multi-Modal Perturbations},
year = {2026},
howpublished = {\url{https://pith.science/paper/Z7JAYGZV}},
note = {Machine review of arXiv:2510.00037}
}
read the original abstract
In Vision-Language-Actionf(VLA) models, robustness to real-world perturbations is critical for deployment. Existing methods target simple visual disturbances, overlooking the broader multi-modal perturbations that arise in actions, instructions, environments, and observations. Here, we first evaluate the robustness of mainstream VLAs under 17 perturbations across four modalities. We find (1) actions as the most fragile modality, (2) Existing visual-robust VLA do not gain robustness in other modality, and (3) pi0 demonstrates superior robustness. To build multi-modal robust VLAs, we propose RobustVLA against perturbations in VLA inputs and outputs. For output robustness, we perform offline robust optimization against worst-case action noise that maximizes mismatch in flow matching objective. This can be seen as adversarial training, label smoothing, and outlier penalization. For input robustness, we enforce consistent actions across input variations that preserve task semantics. To account for multiple perturbations, we formulate robustness as a multi-armed bandit problem and apply an upper confidence bound algorithm to automatically identify the most harmful noise. Experiments on LIBERO demonstrate our RobustVLA delivers absolute gains over baselines of 12.6% on the pi0 backbone and 10.4% on the OpenVLA backbone across all 17 perturbations, achieving 50.6x faster inference than existing visual-robust BYOVLA that requires external LLMs, and a 10.4% gain under mixed perturbations. On the real-world FR5 robot, under four types of multimodal perturbations, RobustVLA shows strong low-data performance, outperforming pi0 by 65.6% success rate with 25 demonstrations. Even with abundant demos, our method still outperform pi0 by 30% success rate. Code and demo videos available at https://github.com/gakakulicc/RobustVLA.
Figures
Forward citations
Cited by 8 Pith papers
-
How VLAs Fail Differently: Black-Box Action Monitoring Reveals Architecture-Specific Failure Signatures
VLA architectures exhibit architecture-specific failure signatures at the motor-command level, with direction reversal as a universal predictor and velocity monitoring ineffective for continuous models.
-
OA-WAM: Object-Addressable World Action Model for Robust Robot Manipulation
OA-WAM uses persistent address vectors and dynamic content vectors in object slots to enable addressable world-action prediction, improving robustness on manipulation benchmarks under scene changes.
-
STRONG-VLA: Decoupled Robustness Learning for Vision-Language-Action Models under Multimodal Perturbations
STRONG-VLA uses decoupled two-stage training to improve VLA model robustness, yielding up to 16% higher task success rates under seen and unseen perturbations on the LIBERO benchmark.
-
Sequential Planning via Anchored Robotic Keypoints
SPARK reaches 43.7% success on six LIBERO-PRO cells by LLM-generated typed behavior trees plus multi-prompt perception and recovery, more than doubling CaP-Agent0 and VLA baselines.
-
APT: Action Expert Pretraining Improves Instruction Generalization of Vision-Language-Action Policies
APT pretrains the action expert as a vision-action prior on frozen VLM features then adds language through gated fusion to improve OOD instruction generalization in continuous-action VLA policies.
-
Breaking the Epistemic Trap: Active Perception Under Compound Uncertainty
The paper defines the Epistemic Trap from interacting state and dynamics uncertainties in RL and introduces an Adaptive Safety Architecture with a Compound Uncertainty Coefficient to enable active information-seeking ...
-
Flatness Preserves Instruction Following in Vision-Language-Action Models
Sharpness-aware minimization during VLA finetuning preserves instruction following and yields over 60% gains across simulation and real-world tasks.
-
Breaking the Epistemic Trap: Active Perception Under Compound Uncertainty
The paper proposes an Adaptive Safety Architecture with a mutual-information-based Compound Uncertainty Coefficient, MaxInfoRL policies, and adaptive constraints to actively resolve compound epistemic uncertainty in R...
Reference graph
Works this paper leans on
-
[1]
Aakriti Agrawal, Rohith Aralikatti, Yanchao Sun, and Furong Huang. Robustness to multi-modal environment uncertainty in marl using curriculum learning.arXiv preprint arXiv:2310.08746,
-
[7]
Flow matching for generative modeling.arXiv preprint arXiv:2210.02747,
Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling.arXiv preprint arXiv:2210.02747,
-
[9]
Towards deep learning models resistant to adversarial attacks.arXiv preprint arXiv:1706.06083,
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks.arXiv preprint arXiv:1706.06083,
-
[11]
Karl Pertsch, Kyle Stachowicz, Brian Ichter, Danny Driess, Suraj Nair, Quan Vuong, Oier Mees, Chelsea Finn, and Sergey Levine. Fast: Efficient action tokenization for vision-language-action models.arXiv preprint arXiv:2501.09747,
-
[12]
Delin Qu, Haoming Song, Qizhi Chen, Yuanqi Yao, Xinyi Ye, Yan Ding, Zhigang Wang, JiaYuan Gu, Bin Zhao, Dong Wang, et al. Spatialvla: Exploring spatial representations for visual-language- action model.arXiv preprint arXiv:2501.15830,
-
[13]
Ranjan Sapkota, Yang Cao, Konstantinos I Roumeliotis, and Manoj Karkee. Vision-language-action models: Concepts, progress, applications and challenges.arXiv preprint arXiv:2505.04769,
-
[14]
Octo: An open-source generalist robot policy.arXiv preprint arXiv:2405.12213,
Octo Model Team, Dibya Ghosh, Homer Walke, Karl Pertsch, Kevin Black, Oier Mees, Sudeep Dasari, Joey Hejna, Tobias Kreiman, Charles Xu, et al. Octo: An open-source generalist robot policy.arXiv preprint arXiv:2405.12213,
-
[15]
Huan Zhang, Hongge Chen, Duane Boning, and Cho-Jui Hsieh. Robust reinforcement learning on state observations with learned optimal adversary.arXiv preprint arXiv:2101.08452,
-
[17]
In robustness against VLA input, we additionally use UCB algorithm to select the best perturbation
Comparing with π0, our RobustVLA gains robustness with minimal loss in clean performance by incorporating robustness against VLA input and output. In robustness against VLA input, we additionally use UCB algorithm to select the best perturbation. Both robustness against VLA input 19 and output are trained using adversarial training with TRADES objective (...
2019
-
[2002]
Gr00t n1: An open foundation model for generalist humanoid robots.arXiv preprint arXiv:2503.14734,
Johan Bjorck, Fernando Castañeda, Nikita Cherniadev, Xingye Da, Runyu Ding, Linxi Fan, Yu Fang, Dieter Fox, Fengyuan Hu, Spencer Huang, et al. Gr00t n1: An open foundation model for generalist humanoid robots.arXiv preprint arXiv:2503.14734,
-
[2017]
Daniel J Mankowitz, Nir Levine, Rae Jeong, Yuanyuan Shi, Jackie Kay, Abbas Abdolmaleki, Jost To- bias Springenberg, Timothy Mann, Todd Hester, and Martin Riedmiller. Robust reinforcement learning for continuous control with model misspecification.arXiv preprint arXiv:1906.07516,
Pith/arXiv arXiv 1906
-
[2020]
Qixiu Li, Yaobo Liang, Zeyu Wang, Lin Luo, Xi Chen, Mozheng Liao, Fangyun Wei, Yu Deng, Sicheng Xu, Yizhong Zhang, et al. Cogact: A foundational vision-language-action model for synergizing cognition and action in robotic manipulation.arXiv preprint arXiv:2411.19650,
-
[2021]
Yifan Zhong, Fengshuo Bai, Shaofei Cai, Xuchuan Huang, Zhang Chen, Xiaowei Zhang, Yuanfei Wang, Shaoyang Guo, Tianrui Guan, Ka Nam Lui, et al. A survey on vision-language-action models: An action tokenization perspective.arXiv preprint arXiv:2507.01925,
-
[2022]
Maximum entropy rl (provably) solves some robust rl problems.arXiv preprint arXiv:2103.06257,
Benjamin Eysenbach and Sergey Levine. Maximum entropy rl (provably) solves some robust rl problems.arXiv preprint arXiv:2103.06257,
-
[2023]
Rectified flow: A marginal preserving approach to optimal transport.arXiv preprint arXiv:2209.14577,
Qiang Liu. Rectified flow: A marginal preserving approach to optimal transport.arXiv preprint arXiv:2209.14577,
-
[2024]
Rt-1: Robotics transformer for real-world control at scale.arXiv preprint arXiv:2212.06817,
Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Jasmine Hsu, et al. Rt-1: Robotics transformer for real-world control at scale.arXiv preprint arXiv:2212.06817,
-
[2025]
Sergey Levine, Aviral Kumar, George Tucker, and Justin Fu. Offline reinforcement learning: Tutorial, review, and perspectives on open problems.arXiv preprint arXiv:2005.01643,
Pith/arXiv arXiv 2005
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.