REVIEW 6 cited by
Run-time Observation Interventions Make Vision-Language-Action Models More Visually Robust
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Vision-language-action (VLA) models trained on large-scale internet data and robot demonstrations have the potential to serve as generalist robot policies. However, despite their large-scale training, VLAs are often brittle to task-irrelevant visual details such as distractor objects or background colors. We introduce Bring Your Own VLA (BYOVLA): a run-time intervention scheme that (1) dynamically identifies regions of the input image that the model is sensitive to, and (2) minimally alters task-irrelevant regions to reduce the model's sensitivity using automated image editing tools. Our approach is compatible with any off the shelf VLA without model fine-tuning or access to the model's weights. Hardware experiments on language-instructed manipulation tasks demonstrate that BYOVLA enables state-of-the-art VLA models to nearly retain their nominal performance in the presence of distractor objects and backgrounds, which otherwise degrade task success rates by up to 40%. Website with additional information, videos, and code: https://aasherh.github.io/byovla/ .
Forward citations
Cited by 6 Pith papers
-
Reimagination with Test-time Observation Interventions: Distractor-Robust World Model Predictions for Visual Model Predictive Control
Removing unfamiliar visual distractors from the input observation before world-model rollout, then compositing them back into predicted frames, makes action verification robust and raises real-robot pick-and-place suc...
-
Adapting by Analogy: OOD Generalization of Visuomotor Policies via Functional Correspondence
A test-time method uses expert-provided functional correspondences to map out-of-distribution scenes to similar training scenes, letting a visuomotor policy reuse old behaviors without retraining.
-
Predictive Red Teaming: Breaking Policies Without Breaking Robots
A generative image editing plus anomaly detection pipeline predicts a visuomotor policy's success-rate degradation across off-nominal environmental factors, with an average prediction error below 0.19 in hardware trials.
-
From Foresight to Forethought: VLM-In-the-Loop Policy Steering via Latent Alignment
FOREWARN steers a diffusion robot policy at runtime by using a world model to predict latent futures and a vision-language model to narrate and rank those futures in natural language.
-
Towards Safe Robot Foundation Models Using Inductive Biases
A modular ATACOM safety layer is added to robot foundation models pi0 and OCTO, yielding provably safe actions with minimal performance loss.
-
Large Model Empowered Embodied AI: A Survey on Decision-Making and Embodied Learning
A review that categorizes large-model-empowered embodied AI into hierarchical and end-to-end decision-making, imitation and reinforcement learning, and world models.
Discussion (0). Continue with ORCID to comment.