Pith. sign in

REVIEW 4 major objections 5 minor 52 references

CofactVLA: Deconfounding Vision-Language-Action Models via Counterfactual Intervention

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read CofactVLA deconfounds vision-language-action policies by subtracting a language-masked counterfactual branch from both latent features and generated actions, and reports a 52.3-point absolute gain in real-world out-of-distribution success.

desk verdict A useful empirical recipe with a load-bearing theoretical flaw: Equation (5) doesn't deconfound, but the experiments deserve referee time. read the letter →

arxiv 2608.04396 v1 pith:7OCYAB4W submitted 2026-08-05 cs.CV

Yan Zhang , Yinan Wu , Haoran Duan , Jungong Han This is my paper · ORCID
classification cs.CV
keywords vision-language-actionmodelscounterfactualinterventionvision-overridephenomenonorthogonalprojectionguidancecovariancereductionflowmatchingout-of-distributiongeneralizationroboticmanipulation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Vision-language-action models often execute the most visually salient action in a scene instead of following the spoken instruction; the paper names this failure the vision-override phenomenon and traces it to a spurious visual-confounder path in the policy's causal graph. CofactVLA responds by adding a language-masked counterfactual branch that predicts what the policy would do from the image alone, and then subtracting that visual-only tendency from both the latent features and the generated velocity field. The paper reports that this dual deconfounding raises the average success rate on a standard simulation benchmark to 98.5 percent, and in real-world tests under unseen conditions raises the base policy's 23.5 percent success rate to 75.8 percent, a 52.3 percentage-point absolute gain. The intended upshot is a general way to make VLA policies follow the language instruction rather than their visual habits, without requiring a new architecture.

What carries the argument

The central object is the language-masked counterfactual branch: the same VLA run in the same visual scene with the language masked out, yielding a visual-only velocity field and visual-only attention features. OPG exploits the exact score-velocity equivalence of the Gaussian flow-matching forward process so that orthogonal projection of the factual velocity field away from the counterfactual one is a legitimate re-weighting of action-mode probabilities rather than a heuristic interpolation. CCR makes the same separation at the representation level by taking the positive part of the covariance difference between the two branches, which the paper proves spans exactly the spurious visual subspace under its gain-bias and contrastive eigengap assumptions.

What would settle it

The decisive experiment is to feed the counterfactual branch a different, semantically meaningful instruction instead of a mask: if the direction of the supposed visual bias changes with the content of that wrong instruction, the branch is not language-free and the subtraction is mis-specified; alternatively, on a dataset where the base policy already follows language with no measurable vision-override, CofactVLA should leave success rates unchanged, and any drop would show the intervention deletes useful signal.

Watch

Extended reading notes

Core claim

On the paper's own terms, action generation in a VLA is a Dual-path Deconfounding Graph: the language instruction has a causal path to the action, but the dense image also opens a backdoor image-to-confounder-to-action path that overrides the instruction. To sever the backdoor, the paper constructs a counterfactual branch in which the language token is masked, treating its output as the pure visual instinct. Action-level OPG projects the factual flow velocity onto that instinct and subtracts the projection, leaving an orthogonal component that is then re-weighted as the deconfounded semantic intent; feature-level CCR takes the covariance difference between counterfactual and factual attention features, extracts its positive eigenspace as the nuisance basis, and projects the features away from it. The paper asserts that this dual intervention cleanly eradicates visual bias while preserving the valid action manifold, and supports the claim with state-of-the-art simulation scores and a 52.3-point absolute real-world OOD improvement over the base policy.

Load-bearing premise

The load-bearing premise is that the language-masked, image-only branch captures exactly the visual confounder effect, with that effect collinear with the no-language velocity field or confined to the positive eigenspace of the covariance difference; if the image-only branch still carries useful semantic or affordance information, subtracting it removes legitimate signal and the deconfounded action is not guaranteed to preserve the language intent.

Editorial extensions

If this is right

  • Policies trained with CofactVLA should keep following the instruction when a salient distractor object competes for the robot's attention, which is the exact failure mode the paper diagnoses.
  • The intervention operates inside a single forward pass, so it can be layered onto existing flow-matching VLA checkpoints instead of requiring a redesigned policy network.
  • The reported zero-shot results imply that the method transfers across simulation and physical hardware: the real-world OOD success rate climbs from 23.5 percent to 75.8 percent while the standard-environment rate stays at 90.8 percent.
  • Because the covariance and projection penalties are applied at two different levels of the stack, the two mechanisms can be ablated, tuned, or combined independently, with the paper reporting combined performance above either alone.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves implicit that the same orthogonal-subtraction recipe applies to any two-modality policy where one dense modality overrides a sparse command, so the method could transfer beyond robotics to vision-and-language generation or audio-visual control.
  • The key untested boundary is the assumption that the image-only branch encodes no task-relevant semantics; a stronger design would estimate the confounder subspace from data instead of identifying it with the language-masked branch.
  • The reported sensitivity sweeps (guidance scale gamma and intervention strength beta) suggest the method's benefit depends on calibrating both strengths, so one testable extension is an automatic scheduler that adapts them per scene rather than using fixed values.
  • If the deconfounding truly isolates a causal language direction, then interventions on the orthogonal component should be composable: the same projection machinery could in principle damp multiple confounders (color, layout, camera angle) one at a time.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The manuscript proposes CofactVLA, a two-branch causal intervention framework for Vision-Language-Action (VLA) models that aims to remove 'vision-override' bias caused by spurious visual confounders. A factual branch conditioned on image and language and a counterfactual branch conditioned only on the image are generated in a single forward pass; the paper then applies two interventions: Action-Level Orthogonal Projection Guidance (OPG) to the flow-matching velocity fields, and Feature-Level Counterfactual Covariance Reduction (CCR) to latent attention features. Experiments on LIBERO, LIBERO-Plus, and a real 6-DoF robot arm report state-of-the-art results, including a 52.3% absolute success-rate gain in real-world out-of-distribution scenarios. The central claim is that these interventions 'cleanly eradicate' visual bias and preserve the pure language semantic intent.

Significance. If the deconfounding mechanism were correct as stated, this would be an important contribution: a general counterfactual intervention method for suppressing visual shortcuts in VLA policies, with strong evidence of OOD generalization. The proposed framework is simple and could be applied to any flow-matching-based VLA. However, the theoretical foundation is the principal added value of the paper, and it is not currently supported by the manuscript's own equations. The empirical improvements may be real, but as written they do not establish the causal deconfounding mechanism claimed. The paper does not ship code or machine-checked proofs, so the credibility of the contribution rests on the derivation and the experiments.

major comments (4)
  1. [Section 3.2, Eq. (5)] The central OPG formula does not perform the claimed deconfounding. Under the paper's own assumption that the counterfactual branch v_uncond captures exactly the visual bias v_bias, decompose v_cond = v_sem + v_bias with v_sem orthogonal to v_bias. Eq. (3) gives v_proj = v_bias and Eq. (4) gives v_perp = v_sem. But Eq. (5) then yields v_causal = v_cond + gamma * v_perp = (1 + gamma) * v_sem + v_bias. The spurious bias term v_bias remains with coefficient 1; it is not eradicated, only relatively downweighted. The claims in Section 3.2 that OPG 'cleanly eradicates visual bias' and 'exclusively re-weights the causal mode odds along the semantic direction' are therefore false as written. To actually remove the bias, the formula would need v_causal = v_perp or v_causal = v_cond - gamma * v_proj. Because OPG is one of the two core contributions, this is a load-bearing error.
  2. [Section 3.2, density definitions] The marginal densities are mislabeled. The manuscript defines 'p_cond := p(A_tau | O)' and 'p_uncond := p(A_tau | T)', but the factual branch defined in Section 3.1 is conditioned on both O and T, and the counterfactual branch is conditioned on O only. The correct labels should be p(A_tau | O, T) and p(A_tau | O), respectively. This reversal is more than a typo: it indicates a sign or definitional inconsistency in the derivation of the intervention, and it should be corrected before the score/velocity composition argument can be evaluated.
  3. [Section 3.3 and Appendix A.1, CCR proof] The proof of Theorem 1 does not support the implemented CCR algorithm. The appendix establishes identifiability for the top generalized eigenspace of the pair (Sigma_Delta, Sigma_0) and uses the Sigma_0-orthogonal projector Pi_C = B(B^T Sigma_0 B)^{-1} B^T Sigma_0. In contrast, the main text Eq. (6)-(9) performs an eigendecomposition of Delta_Sigma = Sigma_cf - Sigma_f and subtracts the plain projection (F U_bias) U_bias^T using ordinary eigenvectors with eigenvalues above epsilon. Assumption 2 is stated for the whitened matrix M = Sigma_0^{-1/2} Sigma_Delta Sigma_0^{-1/2}, not for Delta_Sigma. The proof therefore does not show that the top-k eigenvectors of Delta_Sigma span S_C, nor that the plain projection in Eq. (9) removes the nuisance component while preserving S_O. The theoretical guarantee for CCR is unproved for the implemented method.
  4. [Section 3.3, Assumptions 1-2] The 'strict' guarantees of CCR rest entirely on Assumptions 1 and 2, which are asserted without empirical validation or a concrete geometric argument linking the latent feature space of a VLA backbone to the assumed orthogonal decomposition. In particular, Assumption 2 requires that the eigenvalues of the whitened covariance difference on the confounder subspace are strictly larger than those on the intent subspace; the paper provides no experiment, visualization, or diagnostic that checks this spectral separation. Since the method's causal interpretation depends on this assumption holding in practice, the current manuscript does not establish that CCR actually deconfounds rather than merely applying a heuristic feature-space penalty.
minor comments (5)
  1. [Section 3.2, Eq. (1)] Eq. (1) is the definition of conditional mutual information, not a substantive theorem; the text's phrasing 'This theorem explicitly dictates that the visual confounder intrinsically corrupts' overstates what the identity establishes.
  2. [Figure 2 caption] The caption refers to 'Contrastive Covariance Reduction' while the method is called 'Counterfactual Covariance Reduction' in the text and abstract; the terminology should be unified.
  3. [Table 2] Several entries in Table 2 appear to run together (e.g., '65.121.6' and 'Openvla-OFT_m: 55.6 21.781.0 92.791.0 78.668.7 67.9'), suggesting a formatting or column-alignment error that should be fixed.
  4. [Appendix A.5] There are stray '/uni000...' character sequences in the text around Figure 12 and several figure captions, indicating a corrupted source-rendering artifact that should be removed.
  5. [Contributions and Section 5] The wording 'strictly deconfounded action trajectories' and 'strictly and consistently grounded in causal semantic intents' is stronger than what the current equations support; after correcting OPG, these claims should be reworded to match the implemented formula.

Circularity Check

1 steps flagged · score 6.0 of 10

OPG's 'pure semantic intent' is the residual defined by its own counterfactual branch; Eq. (5) retains v_bias, so the deconfounding claim is built into the definition rather than derived.

  1. self definitional [Section 3.2, Eqs. (3)-(5)]
    "By explicitly subtracting this collinear visual bias, we extract the strictly orthogonal velocity component v⊥, which represents the pure, deconfounded language semantic intent: v⊥ = vcond − vproj. ... The final causally intervened velocity field is then constructed as: vcausal = vcond + γ·v⊥ ... this orthogonalization guarantees that our intervention exclusively re-weights the causal mode odds along the semantic direction, cleanly eradicating visual bias."

    The paper first stipulates that the language-masked branch (conditioned only on O) is the visual confounder, so v_uncond is declared to be the bias. Under that stipulation, Eq. (3) projects v_cond onto the bias and Eq. (4) defines v_perp as whatever remains. Calling that residual the 'pure, deconfounded language semantic intent' is therefore a re-description of the construction, not an empirical or mathematical derivation. The claim of clean eradication also fails under the paper's own decomposition: if v_cond = v_sem + v_bias, then Eq. (5) gives v_causal = (1+γ)v_sem + v_bias, so the bias survives with coefficient 1. The central deconfounding result is thus equivalent to the input definitions and the unproved identification of v_uncond with C.

full rationale

The main derivation chain is not supported by load-bearing self-citations: the only author self-citation ([21], related work on parameter-efficient adaptation) is not load-bearing, and the OOD/real-world evaluations are external benchmarks that would remain informative even if the causal story were wrong. However, the action-level claim that OPG 'cleanly eradicates visual bias and exclusively re-weights the causal mode odds' is not a consequence of Eqs. (3)-(5); it is an interpretation attached to the residual v_perp after the paper has already identified the no-language branch with the confounder. That identification is the entire content of the deconfounding claim, and it is assumed rather than tested. Furthermore, Eq. (5) adds the original factual velocity back, so under the paper's own orthogonal decomposition the bias term remains present, making the 'clean eradication' wording false as written. The scalar strengths γ and β are tuned on the LIBERO benchmark before reporting LIBERO results, which is a fitting concern, but the real-world +52.3% OOD gain is an independent external result, so this is not a fitted-input-called-prediction circularity by itself. On balance: one central step reduces by construction to its own counterfactual-branch definition, warranting a partial circularity score of 6; the empirical results are not made circular by that step.

Assumptions & free parameters 3 free parameters · 4 assumptions · 1 invented entities

The central claim rests on two strong structural assumptions (orthogonal subspace decomposition and spectral separation) plus the interpretation of the counterfactual branch as the pure confounder. The two scalar hyperparameters gamma and beta are tuned on the evaluation benchmark, and the confounder C is an inferred latent variable without independent evidence.

free parameters (3)
  • causal scale gamma = 2.0
    Tuned via sensitivity sweep on LIBERO (Figure 4, Table 4); performance varies with gamma, so the reported results depend on this choice.
  • intervention strength beta = 0.15
    Tuned via sensitivity sweep on LIBERO (Figure 4, Table 4); peak at beta=0.15.
  • eigenvalue threshold epsilon for top-k selection = not reported
    The positive eigenspace is selected with lambda > epsilon > 0 (Section 3.3), but epsilon and k are not specified, making the feature intervention under-determined.
assumptions (4)
  • ad hoc to paper Assumption 1: latent feature space decomposes orthogonally into causal intent subspace S_O and spurious confounder subspace S_C.
    Section 3.3, Assumption 1; no empirical validation is provided for this orthogonal decomposition.
  • ad hoc to paper Assumption 2: eigenvalues of the whitened covariance difference on the confounder subspace are strictly larger than those on the intent subspace.
    Section 3.3, Assumption 2; this spectral separation is what makes Ubias identifiable as the confounder, but it is not derived from data.
  • domain assumption Demonstrations exhibit nuisance-correlated mode selection, I(Z;C|O)>0 for action modes Z=h(A).
    Section 3.2 uses this to argue I(A;C|O)>0; it is asserted without verification on the datasets.
  • domain assumption The language-masked counterfactual branch yields the visual confounder effect.
    Section 3.1: the counterfactual branch conditions only on O, and its output is interpreted as pure visual bias; if the no-language policy still contains useful semantic or affordance information, the subtraction removes legitimate signal.
invented entities (1)
  • Visual confounder C
    purpose: Latent variable representing prominent objects or familiar layouts that spuriously drive actions; the method aims to neutralize it.
    C is inferred from the model's own counterfactual branch; no direct measurement or falsifiable prediction outside the method.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CofactVLA: Deconfounding Vision-Language-Action Models via Counterfactual Intervention." pith.science (2026). https://pith.science/paper/7OCYAB4W

@misc{pith2026260804396,
  author       = {Pith},
  title        = {Pith review of: CofactVLA: Deconfounding Vision-Language-Action Models via Counterfactual Intervention},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7OCYAB4W}},
  note         = {Machine review of arXiv:2608.04396}
}
read the original abstract

Vision-Language-Action (VLA) models have driven significant progress in robotic manipulation, yet they fundamentally struggle with the vision-override phenomenon. Driven by the severe modality imbalance between dense visual streams and sparse linguistic instructions, VLAs frequently fall prey to causal confusion. Instead of treating language as the primary causal driver, the policy entirely bypasses the original instruction by overfitting to spurious visual confounders, such as prominent objects or familiar layouts. To systematically alleviate this bias, we formalize the process of action generation as a Dual-path Deconfounding Graph (DDG) and propose CofactVLA, a novel causal intervention framework. By dynamically constructing a language-masked counterfactual branch within a single forward pass, CofactVLA isolates and neutralizes visual confounders through two synergistic mechanisms. First, Action-Level Orthogonal Projection Guidance (OPG) geometrically projects the factual velocity field away from the counterfactual visual bias during continuous flow matching, extracting the pure semantic intent. Second, Feature-Level Counterfactual Covariance Reduction (CCR) mathematically deconfounds latent representations by penalizing the positive eigenspace of the covariance difference, explicitly suppressing dominant visual shortcuts while preserving the causal language intent. Extensive experiments demonstrate that CofactVLA establishes a new state-of-the-art across diverse simulation benchmarks. Beyond simulation, real-world robot experiments demonstrate the causal efficacy of our method in bridging the generalization gap, yielding a 52.3\% absolute success rate gain under out-of-distribution scenarios.

Figures

Figures reproduced from arXiv: 2608.04396 by the authors.

Figure 1
Figure 1. (a) Motivation: Visual modality inadvertently induces a spurious backdoor path (I 99K C → A). The visual confounder C acts as a shortcut that dictates the action A, overriding the true semantic intent T. (b) Our proposed Dual-path Deconfounding Graph: The red do-operator signifies Feature-Level Counterfactual Covariance Reduction (CCR), which explicitly suppresses the spurious visual confounder C. The blue do-operat… view at source ↗
Figure 2
Figure 2. The overall architecture of CofactVLA. (Left) Driven by the proposed Dual-path Deconfounding Graph (DDG), we propose an intervention framework to address the vision-override phenomenon. (Right) The framework consists of two core components dynamically built upon factual and counterfactual flows. To execute Language Intervention, Orthogonal Projection Guidance (OPG) geometrically extracts the pure semantic intent by … view at source ↗
Figure 4
Figure 4. Sensitivity analyses of intervention intensities. Top row: action-level causal scale γ. Bottom row: feature-level intervention strength β. Put the yellow and white mug in the microwave and close it π0.5 CofactVLA π0.5 CofactVLA Put both moka pots on the stove Pick the red cube into the yellow plate (a) LIBERO Benchmark (b) Real-World Robot Put the apple on the yellow plate [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figures from the paper (16 more)
Figure 5
Figure 5. Figure 5: Qualitative comparison of mitigating the “vision-override” phenomenon on the [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Qualitative comparison of OOD generalization. While π0.5 succeeds on the standard background (left), it fails completely on the unseen checkered texture (right). In contrast, CofactVLA robustly completes the task by decoupling visual confounders. Pick the red cube into…
Figure 7
Figure 7. Figure 7: OOD generalization of CofactVLA: Visualizations from real-world experiments. 4.6 Qualitative Results and Analyses We present several qualitative visualizations in both simulation and real-world environments. Please refer to the Appendix A.6 for more comprehensive quali…
Figure 8
Figure 8. Figure 8: Our hardware platform for the real-world robot experiments. [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]
Figure 9
Figure 9. Figure 9: Example of dual-view sample collection for Task I and Task II. [PITH_FULL_IMAGE:figures/full_fig_p017_9.png]
Figure 10
Figure 10. Figure 10: Example of dual-view sample collection for Task III and Task IV. [PITH_FULL_IMAGE:figures/full_fig_p018_10.png]
Figure 11
Figure 11. Figure 11: Visualizations of the perturbation design for different tasks. [PITH_FULL_IMAGE:figures/full_fig_p019_11.png]
Figure 12
Figure 12. Figure 12: We report success rate (100 trials) for 9 skills (x-axis) and the average over all tasks (left [PITH_FULL_IMAGE:figures/full_fig_p020_12.png]
Figure 13
Figure 13. Figure 13: Qualitative comparison between our CofactVLA model and [PITH_FULL_IMAGE:figures/full_fig_p021_13.png]
Figure 14
Figure 14. Figure 14: Qualitative results of our CofactVLA model on the [PITH_FULL_IMAGE:figures/full_fig_p022_14.png]
Figure 15
Figure 15. Figure 15: Qualitative results of our CofactVLA model on the [PITH_FULL_IMAGE:figures/full_fig_p022_15.png]
Figure 16
Figure 16. Figure 16: Qualitative results of our CofactVLA model on the [PITH_FULL_IMAGE:figures/full_fig_p023_16.png]
Figure 17
Figure 17. Figure 17: Qualitative results of our CofactVLA model on the [PITH_FULL_IMAGE:figures/full_fig_p023_17.png]
Figure 18
Figure 18. Figure 18: Qualitative results of our CofactVLA model on the [PITH_FULL_IMAGE:figures/full_fig_p024_18.png]
Figure 19
Figure 19. Figure 19: Failed results of our CofactVLA model on the [PITH_FULL_IMAGE:figures/full_fig_p025_19.png]
Figure 20
Figure 20. Figure 20: Failed results of our CofactVLA model on the [PITH_FULL_IMAGE:figures/full_fig_p026_20.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

52 extracted references · 24 canonical work pages

  1. [1]

    RT-1: Robotics transformer for real-world control at scale.arXiv preprint arXiv:2212.06817, 2022

    Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Jasmine Hsu, et al. RT-1: Robotics transformer for real-world control at scale.arXiv preprint arXiv:2212.06817, 2022

    arXiv 2022

  2. [2]

    RT-2: Vision-language-action models transfer web knowledge to robotic control

    Brianna Zitkovich, Tianhe Yu, Sichun Xu, Peng Xu, Ted Xiao, Fei Xia, Jialin Wu, Paul Wohlhart, Stefan Welker, Ayzaan Wahid, Quan Vuong, Vincent Vanhoucke, Huong Tran, Radu Soricut, Anikait Singh, Jaspiar Singh, Pierre Sermanet, Pannag R Sanketi, Grecia Salazar, Michael S Ryoo, Krista Reymann, Kanishka Rao, Karl Pertsch, Igor Mordatch, Henryk Michalewski, ...

    work page 2023

  3. [3]

    Vision-language foundation models as effective robot imitators

    Xinghang Li, Minghuan Liu, Hanbo Zhang, Cunjun Yu, Jie Xu, Hongtao Wu, Chilam Cheang, Ya Jing, Weinan Zhang, Huaping Liu, Hang Li, and Tao Kong. Vision-language foundation models as effective robot imitators. InThe Twelfth International Conference on Learning Representations, 2024

    2024

  4. [4]

    Unleashing large-scale video generative pre-training for visual robot manipulation

    Hongtao Wu, Ya Jing, Chilam Cheang, Guangzeng Chen, Jiafeng Xu, Xinghang Li, Minghuan Liu, Hang Li, and Tao Kong. Unleashing large-scale video generative pre-training for visual robot manipulation. In The Twelfth International Conference on Learning Representations, 2024

    work page 2024

  5. [5]

    Gr00t n1: An open foundation model for generalist humanoid robots.arXiv preprint arXiv:2503.14734, 2025

    Johan Bjorck, Fernando Castañeda, Nikita Cherniadev, Xingye Da, Runyu Ding, Linxi Fan, Yu Fang, Dieter Fox, Fengyuan Hu, Spencer Huang, et al. Gr00t n1: An open foundation model for generalist humanoid robots.arXiv preprint arXiv:2503.14734, 2025

    arXiv 2025

  6. [6]

    Moto: Latent motion token as the bridging language for learning robot manipulation from videos

    Yi Chen, Yuying Ge, Weiliang Tang, Yizhuo Li, Yixiao Ge, Mingyu Ding, Ying Shan, and Xihui Liu. Moto: Latent motion token as the bridging language for learning robot manipulation from videos. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 19752–19763, 2025

    work page 2025

  7. [7]

    TraceVLA: Visual trace prompting enhances spatial-temporal awareness for generalist robotic policies

    Ruijie Zheng, Yongyuan Liang, Shuaiyi Huang, Jianfeng Gao, Hal Daumé III, Andrey Kolobov, Furong Huang, and Jianwei Yang. TraceVLA: Visual trace prompting enhances spatial-temporal awareness for generalist robotic policies. InThe Thirteenth International Conference on Learning Representations, 2025

    work page 2025

  8. [8]

    GraspVLA: a grasping foundation model pre-trained on billion-scale synthetic action data

    Shengliang Deng, Mi Yan, Songlin Wei, Haixin Ma, Yuxin Yang, Jiayi Chen, Zhiqi Zhang, Taoyu Yang, Xuheng Zhang, Heming Cui, Zhizheng Zhang, and He Wang. GraspVLA: a grasping foundation model pre-trained on billion-scale synthetic action data. InProceedings of The 9th Conference on Robot Learning, pages 1004–1029. PMLR, 2025

    work page 2025

Show all 52 references
  1. [9]

    Hi Robot: Open-ended instruction following with hierarchical vision- language-action models

    Lucy Xiaoyang Shi, brian ichter, Michael Robert Equi, Liyiming Ke, Karl Pertsch, Quan Vuong, James Tanner, Anna Walling, Haohuan Wang, Niccolo Fusai, Adrian Li-Bell, Danny Driess, Lachy Groom, Sergey Levine, and Chelsea Finn. Hi Robot: Open-ended instruction following with hie...

    2025

  2. [10]

    Libero-plus: In-depth robustness analysis of vision-language-action models.arXiv preprint arXiv:2510.13626, 2025

    Senyu Fei, Siyin Wang, Junhao Shi, Zihao Dai, Jikun Cai, Pengfang Qian, Li Ji, Xinzhe He, Shiduo Zhang, Zhaoye Fei, et al. Libero-plus: In-depth robustness analysis of vision-language-action models.arXiv preprint arXiv:2510.13626, 2025

    arXiv

  3. [11]

    Cast: Counterfactual labels improve instruction following in vision-language-action models.arXiv preprint arXiv:2508.13446, 2025

    Catherine Glossop, William Chen, Arjun Bhorkar, Dhruv Shah, and Sergey Levine. Cast: Counterfactual labels improve instruction following in vision-language-action models.arXiv preprint arXiv:2508.13446, 2025

    2025 arXiv

  4. [12]

    Counterfactual vla: Self-reflective vision-language-action model with adaptive reasoning.arXiv preprint arXiv:2512.24426, 2025

    Zhenghao Peng, Wenhao Ding, Yurong You, Yuxiao Chen, Wenjie Luo, Thomas Tian, Yulong Cao, Apoorva Sharma, Danfei Xu, Boris Ivanovic, et al. Counterfactual vla: Self-reflective vision-language-action model with adaptive reasoning.arXiv preprint arXiv:2512.24426, 2025. 10

    2025

  5. [13]

    Classifier-free diffusion guidance.arXiv preprint arXiv:2207.12598, 2022

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance.arXiv preprint arXiv:2207.12598, 2022

    2022 arXiv

  6. [14]

    OpenVLA: An open-source vision-language-action model.arXiv preprint arXiv:2406.09246, 2024

    Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, Quan Vuong, Thomas Kollar, Benjamin Burchfiel, Russ Tedrake, Dorsa Sadigh, Sergey Levine, Percy Liang, and Chelsea Finn. OpenVLA...

    2024 arXiv

  7. [15]

    SimpleVLA-RL: Scaling VLA training via reinforcement learning

    Haozhan Li, Yuxin Zuo, Jiale Yu, Yuhao Zhang, Yang Zhaohui, Kaiyan Zhang, Xuekai Zhu, Yuchen Zhang, Tianxing Chen, Ganqu Cui, Dehui Wang, Dingxiang Luo, Yuchen Fan, Youbang Sun, Jia Zeng, Jiangmiao Pang, Shanghang Zhang, Yu Wang, Yao Mu, Bowen Zhou, and Ning Ding. SimpleVLA-RL...

    2026

  8. [16]

    SmolVLA: A vision-language- action model for affordable and efficient robotics.arXiv preprint arXiv:2506.01844, 2025

    Mustafa Shukor, Dana Aubakirova, Francesco Capuano, Pepijn Kooijmans, Steven Palma, Adil Zouitine, Michel Aractingi, Caroline Pascal, Martino Russi, Andres Marafioti, et al. SmolVLA: A vision-language- action model for affordable and efficient robotics.arXiv preprint arXiv:250...

    2025 arXiv

  9. [17]

    π0.5: a vision-language-action model with open-world generalization.arXiv preprint arXiv:2504.16054, 2025

    Physical Intelligence, Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, et al. π0.5: a vision-language-action model with open-world generalization.arXiv preprint arXiv:2504.16054, 2025

    2025 arXiv

  10. [18]

    Fine-tuning vision-language-action models: Optimizing speed and success.arXiv preprint arXiv:2502.19645, 2025

    Moo Jin Kim, Chelsea Finn, and Percy Liang. Fine-tuning vision-language-action models: Optimizing speed and success.arXiv preprint arXiv:2502.19645, 2025

    2025 arXiv

  11. [19]

    LoRA: Low-rank adaptation of large language models

    Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. InInternational Conference on Learning Representations, 2022

    2022

  12. [20]

    Vla-adapter: An effective paradigm for tiny-scale vision-language-action model

    Yihao Wang, Pengxiang Ding, Lingxiao Li, Can Cui, Zirui Ge, Xinyang Tong, Wenxuan Song, Han Zhao, Wei Zhao, Pengxu Hou, et al. Vla-adapter: An effective paradigm for tiny-scale vision-language-action model. InProceedings of the AAAI Conference on Artificial Intelligence, pages...

    2026

  13. [21]

    Modality-experts coordinated adaptation for large multimodal models.Science China Information Sciences, 67(12):220107, 2024

    Yan Zhang, Zhong Ji, Yanwei Pang, Jungong Han, and Xuelong Li. Modality-experts coordinated adaptation for large multimodal models.Science China Information Sciences, 67(12):220107, 2024

    2024

  14. [22]

    Reconvla: Reconstructive vision-language-action model as effective robot perceiver

    Wenxuan Song, Ziyang Zhou, Han Zhao, Jiayi Chen, Pengxiang Ding, Haodong Yan, Yuxin Huang, Feilong Tang, Donglin Wang, and Haoang Li. Reconvla: Reconstructive vision-language-action model as effective robot perceiver. InProceedings of the AAAI Conference on Artificial Intellig...

    2026

  15. [23]

    Ren, Homer Walke, Quan Vuong, Lucy Xiaoyang Shi, and Sergey Levine

    Danny Driess, Jost Tobias Springenberg, brian ichter, LILI YU, Adrian Li-Bell, Karl Pertsch, Allen Z. Ren, Homer Walke, Quan Vuong, Lucy Xiaoyang Shi, and Sergey Levine. Knowledge insulating vision- language-action models: Train fast, run fast, generalize better. InThe Thirty-...

    2025

  16. [24]

    Mole-vla: Dynamic layer-skipping vision language action model via mixture-of-layers for efficient robot manipulation

    Rongyu Zhang, Menghang Dong, Yuan Zhang, Liang Heng, Xiaowei Chi, Gaole Dai, Li Du, Dan Wang, Yuan Du, and Shanghang Zhang. Mole-vla: Dynamic layer-skipping vision language action model via mixture-of-layers for efficient robot manipulation. InProceedings of the AAAI Conferenc...

    2026

  17. [25]

    Spec-vla: speculative decoding for vision-language-action models with relaxed acceptance

    Songsheng Wang, Rucheng Yu, Zhihang Yuan, Chao Yu, Feng Gao, Yu Wang, and Derek F Wong. Spec-vla: speculative decoding for vision-language-action models with relaxed acceptance. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 269...

    2025

  18. [26]

    CogVLA: Cognition-aligned vision-language- action models via instruction-driven routing & sparsification

    Wei Li, Renshan Zhang, Rui Shao, Jie He, and Liqiang Nie. CogVLA: Cognition-aligned vision-language- action models via instruction-driven routing & sparsification. InThe Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025

    2025

  19. [27]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. InAdvances in Neural Information Processing Systems, pages 6840–6851, 2020

    2020

  20. [28]

    Diffusion policy: Visuomotor policy learning via action diffusion.The International Journal of Robotics Research, 44(10-11):1684–1704, 2025

    Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion.The International Journal of Robotics Research, 44(10-11):1684–1704, 2025

    2025

  21. [29]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 4195–4205, 2023. 11

    2023

  22. [30]

    Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. Flow matching for generative modeling. InThe Eleventh International Conference on Learning Representations, 2023

    2023

  23. [31]

    Flow straight and fast: Learning to generate and transfer data with rectified flow

    Xingchao Liu, Chengyue Gong, and qiang liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. InThe Eleventh International Conference on Learning Representations, 2023

    2023

  24. [32]

    RDT-1b: a diffusion foundation model for bimanual manipulation

    Songming Liu, Lingxuan Wu, Bangguo Li, Hengkai Tan, Huayu Chen, Zhengyi Wang, Ke Xu, Hang Su, and Jun Zhu. RDT-1b: a diffusion foundation model for bimanual manipulation. InThe Thirteenth International Conference on Learning Representations, 2025

    2025

  25. [33]

    One-step diffusion policy: Fast visuomotor policies via diffusion distillation

    Zhendong Wang, Max Li, Ajay Mandlekar, Zhenjia Xu, Jiaojiao Fan, Yashraj Narang, Linxi Fan, Yuke Zhu, Yogesh Balaji, Mingyuan Zhou, Ming-Yu Liu, and Yu Zeng. One-step diffusion policy: Fast visuomotor policies via diffusion distillation. InForty-second International Conference...

    2025

  26. [34]

    π0: A vision-language-action flow model for general robot control.arXiv preprint arXiv:2410.24164, 2024

    Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, et al. π0: A vision-language-action flow model for general robot control.arXiv preprint arXiv:2410.24164, 2024

    arXiv

  27. [35]

    Hybridvla: Collaborative diffusion and autoregression in a unified vision-language-action model.arXiv preprint arXiv:2503.10631, 2025

    Jiaming Liu, Hao Chen, Pengju An, Zhuoyang Liu, Renrui Zhang, Chenyang Gu, Xiaoqi Li, Ziyu Guo, Sixiang Chen, Mengzhen Liu, et al. Hybridvla: Collaborative diffusion and autoregression in a unified vision-language-action model.arXiv preprint arXiv:2503.10631, 2025

    2025 arXiv

  28. [36]

    Toward causal representation learning.Proceedings of the IEEE, 109(5):612– 634, 2021

    Bernhard Schölkopf, Francesco Locatello, Stefan Bauer, Nan Rosemary Ke, Nal Kalchbrenner, Anirudh Goyal, and Yoshua Bengio. Toward causal representation learning.Proceedings of the IEEE, 109(5):612– 634, 2021

    2021

  29. [37]

    Robust agents learn causal world models

    Jonathan Richens and Tom Everitt. Robust agents learn causal world models. InThe Twelfth International Conference on Learning Representations, 2024

    2024

  30. [38]

    Causalworld: A robotic manipulation benchmark for causal structure and transfer learning

    Ossama Ahmed, Frederik Träuble, Anirudh Goyal, Alexander Neitz, Manuel Wuthrich, Yoshua Bengio, Bernhard Schölkopf, and Stefan Bauer. Causalworld: A robotic manipulation benchmark for causal structure and transfer learning. InInternational Conference on Learning Representations, 2021

    2021

  31. [39]

    CDP: Towards robust autoregressive visuomotor policy learning via causal diffusion

    Jiahua Ma, Yiran Qin, Yixiong Li, Xuanqi Liao, Yulan Guo, and Ruimao Zhang. CDP: Towards robust autoregressive visuomotor policy learning via causal diffusion. InProceedings of the 9th Conference on Robot Learning (CoRL), pages 4190–4205. PMLR, 2025

    2025

  32. [40]

    When vision overrides language: Evaluating and mitigating counterfactual failures in vlas.arXiv preprint arXiv:2602.17659, 2026

    Yu Fang, Yuchun Feng, Dong Jing, Jiaqi Liu, Yue Yang, Zhenyu Wei, Daniel Szafir, and Mingyu Ding. When vision overrides language: Evaluating and mitigating counterfactual failures in vlas.arXiv preprint arXiv:2602.17659, 2026

    2026 arXiv

  33. [41]

    Libero: Benchmarking knowledge transfer for lifelong robot learning

    Bo Liu, Yifeng Zhu, Chongkai Gao, Yihao Feng, Qiang Liu, Yuke Zhu, and Peter Stone. Libero: Benchmarking knowledge transfer for lifelong robot learning. InThe Thirty-seventh Annual Conference on Neural Information Processing Systems, 2023

    2023

  34. [42]

    DreamVLA: A vision-language-action model dreamed with comprehensive world knowledge

    Wenyao Zhang, Hongsi Liu, Zekun Qi, Yunnan Wang, XinQiang Yu, Jiazhao Zhang, Runpei Dong, Jiawei He, He Wang, Zhizheng Zhang, Li Yi, Wenjun Zeng, and Xin Jin. DreamVLA: A vision-language-action model dreamed with comprehensive world knowledge. InThe Thirty-ninth Annual Confere...

    2025

  35. [43]

    X-VLA: Soft-prompted transformer as scalable cross-embodiment vision-language-action model

    Jinliang Zheng, Jianxiong Li, Zhihao Wang, Dongxiu Liu, Xirui Kang, Yuchun Feng, Yinan Zheng, Jiayin Zou, Yilun Chen, Jia Zeng, Tai Wang, Ya-Qin Zhang, Jingjing Liu, and Xianyuan Zhan. X-VLA: Soft-prompted transformer as scalable cross-embodiment vision-language-action model. ...

    2026

  36. [44]

    NORA: A small open-sourced generalist vision language action model for embodied tasks.arXiv preprint arXiv:2504.19854, 2025

    Chia-Yu Hung, Qi Sun, Pengfei Hong, Amir Zadeh, Chuan Li, U Tan, Navonil Majumder, Soujanya Poria, et al. NORA: A small open-sourced generalist vision language action model for embodied tasks.arXiv preprint arXiv:2504.19854, 2025

    2025 arXiv

  37. [45]

    Worldvla: Towards autoregressive action world model.arXiv preprint arXiv:2506.21539, 2025

    Jun Cen, Chaohui Yu, Hangjie Yuan, Yuming Jiang, Siteng Huang, Jiayan Guo, Xin Li, Yibing Song, Hao Luo, Fan Wang, et al. Worldvla: Towards autoregressive action world model.arXiv preprint arXiv:2506.21539, 2025

    2025 arXiv

  38. [46]

    UniVLA: Learning to act anywhere with task-centric latent actions.arXiv preprint arXiv:2505.06111, 2025

    Qingwen Bu, Yanting Yang, Jisong Cai, Shenyuan Gao, Guanghui Ren, Maoqing Yao, Ping Luo, and Hongyang Li. UniVLA: Learning to act anywhere with task-centric latent actions.arXiv preprint arXiv:2505.06111, 2025. 12

    2025 arXiv

  39. [47]

    Fast: Efficient action tokenization for vision-language-action models.arXiv preprint arXiv:2501.09747, 2025

    Karl Pertsch, Kyle Stachowicz, Brian Ichter, Danny Driess, Suraj Nair, Quan Vuong, Oier Mees, Chelsea Finn, and Sergey Levine. Fast: Efficient action tokenization for vision-language-action models.arXiv preprint arXiv:2501.09747, 2025

    2025 arXiv

  40. [48]

    Chance- constrained flow matching for high-fidelity constraint-aware generation.arXiv preprint arXiv:2509.25157, 2025

    Jinhao Liang, Yixuan Sun, Anirban Samaddar, Sandeep Madireddy, and Ferdinando Fioretto. Chance- constrained flow matching for high-fidelity constraint-aware generation.arXiv preprint arXiv:2509.25157, 2025

    2025

  41. [49]

    Ddat: Diffusion policies enforcing dynamically admissible robot trajectories.arXiv preprint arXiv:2502.15043, 2025

    Jean-Baptiste Bouvier, Kanghyun Ryu, Kartik Nagpal, Qiayuan Liao, Koushil Sreenath, and Negar Mehr. Ddat: Diffusion policies enforcing dynamically admissible robot trajectories.arXiv preprint arXiv:2502.15043, 2025

    2025 arXiv

  42. [50]

    LeRobot: An open- source library for end-to-end robot learning

    Remi Cadene, Simon Alibert, Francesco Capuano, Michel Aractingi, Adil Zouitine, Pepijn Kooijmans, Jade Choghari, Martino Russi, Caroline Pascal, Steven Palma, Dana Aubakirova, Mustafa Shukor, Jess Moss, Alexander Soare, Quentin Lhoest, Quentin Gallouédec, and Thomas Wolf. LeRo...

    2026

  43. [51]

    put ... on

    Xueyang Zhou, Yangming Xu, Guiyao Tie, Yongchao Chen, Guowen Zhang, Duanfeng Chu, Pan Zhou, and Lichao Sun. Libero-pro: Towards robust and fair evaluation of vision-language-action models beyond memorization.arXiv preprint arXiv:2510.03827, 2025. 13 A Technical appendices and ...

    arXiv

  44. [52]

    remove the cuboid from the blue plate

    combined with drastic camera angle shifts), occasionally causing the vision encoder to lose spatial tracking. In the real-world deployments, execution failures are predominantly observed under severe 21 Pick up the black bowl on the wooden cabinet and place it on the plate Cam...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.