Robotics · arXiv:2610.37509 v2 · submitted 27 Oct 2026 · 60 references
Ego-CEVR: Learning Cross-Embodiment Visual Representations from Human Egocentric Videos
The paper says
“As egocentric video becomes increasingly available for embodied learning, an important research question is how to turn this low-cost, scalable source of data into robot manipulation capabilities.”
more of the abstract
Egocentric videos naturally provide rich context for high-level task understanding, while real-robot demonstrations with wrist observations and action annotations provide supervision for fine-grained action alignment. We present Ego-CEVR, a data-native framework for hierarchical visual representation learning that assigns distinct learning objectives to global and interaction representations. The global module learns object relations, task state, and temporal progress through future prediction on egocentric video. The Interaction module interprets wrist observations in global context and learns local alignment and contact cues through robot action supervision. Separate residual paths expose both representations to the action policy, preserving direct access to task context alongside fine-grained execution information. Ego-CEVR supports independent pretraining of its global video module on large-scale human egocentric videos and videos from diverse robot embodiments, without requiring robot action labels or paired wrist observations.
from the abstract
Pith says
Machine review · deepseek-v4-flash · 30 Oct 2026 · what was read was not recorded · not checked by a person
It can be wrong. Challenge this review or reply to it.
Holds up, with conditions. A coherent hierarchical representation method whose real-robot transfer evidence is too thin to carry the cross-embodiment claim, but the simulation ablations and honest framing make it worth an engaged referee. This line is wrong
4 major objections · 4 minor
How often objections from reviews like this one are right: not yet measured
Not checked: figures, code, what the 60 references say.
The paper says, at §4.4, Tables 5 and 6
The review did not quote this passage.
Pith says: Objection 1 of 4 · major
The central cross-embodiment transfer claim rests entirely on success-rate differences reported without trial counts: Plug Insertion 20→30, Water Pouring 60→75, Package Sorting 60→80, G2 Task 2 0→10, and G1 Task 1 5→15. At typical trial counts of 10-20 per condition, most of these deltas are within binomial sampling noise (for example, 4/20 vs 6/20 on Plug Insertion).
more of objection 1
Please report the number of trials per condition, confidence intervals or raw per-episode logs, and, where feasible, additional trials for the small-delta conditions. Without this information, the conclusion in Section 4.4 and the Conclusion that human egocentric video transfers manipulation knowledge to robot control is not established.
The paper says, at §4.4, Tables 5 and 6
The review did not quote this passage.
Pith says: Objection 2 of 4 · major
The design does not isolate the cross-embodiment contribution of human video. In Table 5, the same-duration Robot Head condition outperforms Human Ego on all three tasks (35/95/85 vs 30/75/80); in Table 6, same-embodiment head video outperforms human ego video in both transfer settings (20 vs 10 on G2 Task 2; 35 vs 15 on G1 Task 1).
more of objection 2
The observed pattern is therefore also consistent with 'any additional head or ego video helps the policy,' rather than with a specifically human-to-robot transfer mechanism. Please add a matched comparison (for example, equal frame counts or equal task coverage between human and robot video sources) or a direct analysis of G_t showing that human and robot head videos of the same task share latent structure that different-task videos do not.
The paper says, at §3.2
The review did not quote this passage.
Pith says: Objection 3 of 4 · major
The paper asserts that L_pred trains the video backbone to produce global representations capturing task state, object relations, and temporal progress, and that these are embodiment-invariant, but G_t is only ever evaluated through downstream success rates. Neither premise is measured directly.
more of objection 3
A linear probe, CKA, or nearest-neighbor retrieval analysis comparing G_t across human ego, robot head, same-task, and different-task videos would directly test the invariance claim and would substantially strengthen the paper. This analysis is also relevant to the scaling discussion in Section 5, since the value of large-scale human video pretraining depends on that invariance.
The paper says, at §4.2, Tables 1-3
The review did not quote this passage.
Pith says: Objection 4 of 4 · major
The headline simulation comparisons are made against numbers reported in separate papers, and several supporting deltas are small: on LIBERO, Ego-CEVR's 97.7% sits within 0.1-0.8 points of the listed baselines, and on RoboTwin Hard the 70.7% is 3.1 points above UniWAM.
more of objection 4
Without multiple seeds or confidence intervals it is not possible to tell whether these differences are meaningful. Please report mean over seeds with variance for Ego-CEVR and for the controlled variants in Table 4, and state explicitly in the text or captions which baseline numbers were re-run under the same protocol and which are quoted from their source papers.
4 minor objections
The paper says, at Figure 2
Pith says: minor objection 1 of 4
The figure labels 'Visuallatens', 'Wristvisualtokens', 'Wristtokens', and 'Interactiontokens' are missing spaces, and the notation for G_t and I_t is rendered inconsistently with subscripts; please clean up these labels.
The paper says, at Table 1 caption
Pith says: minor objection 2 of 4
The caption sentence 'Avg. retains the aggregate reported with each result' is unclear; please specify how each average is computed and which values are literature-reported rather than recomputed.
show 2 more
The paper says, at Table 4
Pith says: minor objection 3 of 4
The quoted Fast-WAM row shows a sharp contrast between its LIBERO average (97.6) and its LIBERO-Plus average (41.3); please clarify whether the quoted baselines were evaluated under the same LIBERO-Plus protocol used for Ego-CEVR, namely a policy trained on LIBERO and evaluated on the perturbed suites.
The paper says, at References
Pith says: minor objection 4 of 4
Reference [14] lacks a venue and is marked only as a project website; please complete the bibliographic entry.
What would settle it
Request the per-trial outcome logs behind Tables 5 and 6 (successes and total trials per cell). Compute exact binomial 95% confidence intervals and Fisher exact tests for each human-ego condition against its no-video baseline: Plug Insertion 30 vs 20, Water Pouring 75 vs 60, Package Sorting 80 vs 60, G2 Task 2 10 vs 0, and G1 Task 1 15 vs 5. If the 95% intervals overlap or the two-sided p-value is at least 0.05 for every comparison, the cross-embodiment transfer claim is not supported at the reported sample size; also state the trial count needed to detect each observed effect size at 80% power. If the authors supply these data and the differences survive, the concern is resolved.
from the skeptic's reading
How the review went
4 readings by deepseek-v4-flash · 30 Oct 2026
- Reader holds up, with conditions
- Skeptic agrees
- Editor wrote the verdict above
- Referee major revision: 4 major objections, 4 minor
Not run on this paper: the simulated authors' defence.
Pith did not check this review's objections against the paper.
The review in full
The review in brief, machine-written
The useful idea here is the division of labor: a global representation learned by future-video prediction on head/ego views, an interaction representation learned by action supervision on wrist views conditioned on the global tokens, and both fed to the action expert through separate residual cross-attention paths. That split is genuinely not in the cited prior work, and it is well motivated: video prediction gives task-level context, action supervision gives local control, and the architecture makes the dependency explicit. The flat-fusion versus hierarchy ablation in Table 4 is the cleanest evidence in the paper, and it lands -- hierarchy buys little on LIBERO in-distribution but a large margin under LIBERO-Plus perturbations, which is the right place for this design to pay off.
The paper's own limitations are stated honestly: no large-scale pretraining was done, so the scaling motivation is untested, and the authors say so directly in Section 5. That is not the soft spot. The soft spot is that the cross-embodiment transfer claim -- the title claim -- rests on Tables 5 and 6, which report success-rate percentages with no trial counts and no confidence intervals. Plug Insertion at 20% to 30%, Task 2 transfer at 0% to 10%, G1 at 5% to 15%: with the trial counts typical of real-robot evaluations, those are within binomial noise. The same-duration robot-head control is also consistently better than human egocentric video, which does not invalidate the method but means the specific human-video contribution is weak and the embodiment-invariance premise is never directly measured. There is no linear probe or similarity analysis showing that human and robot head videos of the same task share a latent that different tasks do not. So the central mechanism -- that future prediction on human ego video builds a transferable embodiment-invariant representation -- is supported only by underpowered success-rate deltas.
I would not hold the comparison style against the paper harshly: comparing to reported numbers is standard in this literature, and the in-house ablations are the controlled part. But the RoboTwin claim of being best on Hard among listed methods is a comparison to other papers' runs, so treat it as informative, not established.
This deserves a serious referee. The method is coherent, the ablation matches the claim, and the honest limitation section is exactly what I want to see. The referee should push for trial-level real-robot data, a direct representation-invariance check, and released code or data. With the transfer evidence made quantitative, this would be a solid contribution to a crowded line.
The strongest objection
The central claim is that future prediction on human egocentric video builds a transferable, embodiment-invariant task-level representation (Sections 3.2 and 4.4). The decisive evidence is Tables 5 and 6, which give only percentages with no number of trials and no confidence intervals. In Table 5, the human-ego increments over Robot Head only are 30 vs 20, 75 vs 60, and 80 vs 60; in Table 6, the increments are 10 vs 0 on G2 Task 2 and 15 vs 5 on G1 Task 1. If the underlying trial counts are 10-20, most of these differences are within binomial sampling noise (for example, 4/20 vs 6/20 on Plug Insertion). The reported same-duration robot-head control produces larger gains than human egocentric video, so the specific cross-embodiment contribution of human video is not isolated. The paper also provides no direct measurement of G_t (linear probe, CKA, or similar) showing that human and robot head videos of the same task share a latent that different-task videos do not. Thus the success-rate deltas are the only evidence for the invariance premise, and they are statistically underdetermined. The Section 5 limitation about not pretraining at scale is separate and honestly acknowledged; the unresolved issue is whether the reported transfer evidence itself supports the claim.
Ego-CEVR is a coherent method paper: the hierarchical split between global context and interaction details is well motivated, the ablations are matched, and the zero-initialized residual action-conditioning branches are a clean way to expose both representations without disturbing the base policy. The central claim, however, is not only that this architecture works but that human egocentric video transfers task-level knowledge across embodiments. That claim requires (i) a shared task-level latent across human ego and robot head views, (ii) G_t encoding that latent, and (iii) real-robot success gains attributable to it. The paper supports (iii) only through Tables 5 and 6, which omit trial counts and confidence intervals; the observed differences are exactly the size one would expect from binomial noise at small sample sizes. The same-duration robot-head control outperforms the human-ego condition, so the human-specific transfer signal is not cleanly isolated. Point (ii) is never checked directly, leaving the reader's invariance premise as an assumption rather than a measured property. This is the most load-bearing weakness because it sits between the method's motivation and its headline empirical result. The Section 5 scaling limitation is real but honestly acknowledged, so it is not the basis of this concern. I agree with the reader's weakest assumption and find the trial-level statistical gap to be the concrete way to settle it. Since the reader already issued a CONDITIONAL verdict and this concern is a sharpening of that condition rather than a reason to reject the method, I would leave the verdict unchanged.
The referee's report
The framework is coherent, and the central design choice — distinct learning objectives for a global, video-supervised representation and a local, action-supervised wrist representation, connected through explicit conditioning — is clean and well motivated. The controlled ablation in Table 4 is the paper's strongest internal evidence: with matched video sources and robot demonstrations, hierarchical conditioning raises the LIBERO-Plus average from 73.8% to 90.2% and camera-perturbation success from 66.7% to 91.1% relative to flat fusion, a large robustness gain consistent with the claimed mechanism. The staged training recipe, in which the global module is pretrained on action-free egocentric video and later grounded by robot action supervision, is practically valuable, and the method has a modest number of free hyperparameters (λ_pred, horizon h, residual scales α and β), with α and β zero-initialized. The Section 5 limitation about the absence of large-scale pretraining is honestly stated and does not by itself block the contribution.
Pith's reading of the paper (a machine's summary, not the authors')
The paper works to turn low-cost human egocentric video, which carries no robot action labels, into a usable source of task knowledge for robot manipulation. Its central claim is that training a video model to predict future frames from head or egocentric observations produces a global representation of object relations, task state, and temporal progress that transfers across embodiments, while a separate wrist-view module supervised by robot actions converts that context into precise local control. If the claim holds, unlabeled video from human daily life could supply the task-understanding half of a manipulation policy, with only a small number of target-robot demonstrations needed to ground execution. Supporting evidence includes the highest listed overall score on LIBERO-Plus (90.2%), the highest listed Hard-setting score on RoboTwin 2.0 (70.7%), and real-robot gains from added human video on Plug Insertion, Water Pouring, and Package Sorting.
What carries the argument. Three components carry the argument. Global tokens $G_t$ are read out from cross-layer attention values of the frozen Wan2.2 video diffusion transformer, which is trained by conditional flow matching to predict future video latents from the current head or egocentric frame plus the language instruction; at inference no future video is generated. Interaction tokens are formed as $I_t = F_I(W_t, G_t)$, with wrist tokens cross-attending to the global tokens under the robot action objective. The action expert is the $\pi_{0.5}$ flow-matching policy augmented by two residual cross-attention branches, one reading $G_t$ and one reading $I_t$, whose learnable scales start at zero so the model begins as the base policy and grows the hierarchical conditioning during training. The joint objective $L_{\text{total}} = L_{\text{act}} + \lambda_{\text{pred}} L_{\text{pred}}$ ties each representation level to its own supervision source: video prediction for global context, robot actions for interaction.
Core claim. The paper's central claim is that one predictive objective can serve two representational needs if the views are separated: future-video prediction on the head or egocentric view yields a global representation $G_t$ carrying task context, while robot action supervision on wrist observations yields an interaction representation $I_t = F_I(W_t, G_t)$ that draws on $G_t$ for scene and goal context and carries local alignment and contact cues. The hierarchy is deliberate: $G_t$ is computed without wrist information and $I_t$ depends on it, so task understanding is learned from cheap, abundant, unlabeled video and then grounded in control by action-labeled robot demonstrations. The authors support the claim with ablations showing the hierarchy beats flat feature fusion mainly under visual perturbations (camera-viewpoint success on LIBERO-Plus rises from 66.7% to 91.1%), that predicting wrist video in addition to head video hurts robustness, and that human egocentric video alone transfers to a new task (0% to 10% success) and, with five target-robot demonstrations, to a new embodiment (5% to 15% success).
The premise it rests on. The load-bearing premise is that future-video prediction over head and egocentric observations captures a task-level latent shared across human and robot embodiments; the paper never measures that invariance directly, only downstream success rates, and it acknowledges that scaling to large video collections was not tested.
If the paper is right (Pith's inferences, not the authors')
Large unlabeled egocentric video collections could serve as the task-understanding pretraining source for manipulation policies, cutting the volume of action-labeled robot data needed.
Head-view prediction plus wrist-view action supervision is a more robust assignment of learning objectives than predicting both views, since joint head/wrist prediction lowered LIBERO-Plus robustness in the ablation.
A few demonstrations from a target robot can align video-learned representations with local control, converting human video (5% to 15% G1 success) or target-robot head video (5% to 35% G1 success) into meaningful cross-embodiment transfer.
The largest and most consistent gains occur under visual perturbations, particularly camera viewpoint changes, where the hierarchical design outperforms flat fusion by 24.4 percentage points.
Because the dual conditioning branches start at zero scale, the representation scheme could be attached to other vision-language-action bases without disturbing their initial behavior.
Inference: because human video nearly matched robot head video on the five-finger Package Sorting task but lagged on Plug Insertion, the authors' data hint that human egocentric video transfers best when the robot's end effector resembles a human hand; that dependency is left untested.
Inference: with only ten hours of video used, the framework's scaling claim is untested; if the mechanism is real, gains should grow monotonically with video hours and show up most on tasks rich in object-state transitions rather than on already-saturated benchmarks.
Inference: the paper attributes robustness gains to the global representation carrying goal and scene context; a natural test it does not run is ablating the $G_t$ branch alone under camera perturbations to confirm the gain comes from task context rather than from the added parameters.
Inference: the claimed cross-embodiment invariance of the predictive representation is never measured directly; probing $G_t$ with task-progress classifiers on matched human and robot videos would settle whether the transfer mechanism is representation-level or task-level.
Check this review
Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
How Pith checks papers · The full audit of this paper · Review history (2 versions)
The paper
Abstract
As egocentric video becomes increasingly available for embodied learning, an important research question is how to turn this low-cost, scalable source of data into robot manipulation capabilities. Egocentric videos naturally provide rich context for high-level task understanding, while real-robot demonstrations with wrist observations and action annotations provide supervision for fine-grained action alignment. We present Ego-CEVR, a data-native framework for hierarchical visual representation learning that assigns distinct learning objectives to global and interaction representations. The global module learns object relations, task state, and temporal progress through future prediction on egocentric video. The Interaction module interprets wrist observations in global context and learns local alignment and contact cues through robot action supervision. Separate residual paths expose both representations to the action policy, preserving direct access to task context alongside fine-grained execution information. Ego-CEVR supports independent pretraining of its global video module on large-scale human egocentric videos and videos from diverse robot embodiments, without requiring robot action labels or paired wrist observations.
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-10-28
The authors' main claim, as the classifier picked it out 0.73
“Ego-CEVR supports independent pretraining of its global video module on large-scale human egocentric videos and videos from diverse robot embodiments, without requiring robot action labels or paired wrist observations.”
How the main claim is stated: asserted 0.98
What the abstract itself states, as probabilities: claims first 0.04 · states limitation 0.13 · human participants 0.34 · controlled comparison 0.04 · randomized assignment 0.01 · states quantitative result 0.03 · compares with prior results 0.05 · mentions code or data availability 0.04
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| As egocentric video becomes increasingly available for embodied learning, an important research question is how to turn this low-cost, scalable source of data into robot manipulation capabilities. | background 0.96 | 0.05 | 0.04 | asserted 0.90 |
| Egocentric videos naturally provide rich context for high-level task understanding, while real-robot demonstrations with wrist observations and action annotations provide supervision for fine-grained action alignment. | background 0.95 | 0.11 | 0.05 | asserted 1.00 |
| We present Ego-CEVR, a data-native framework for hierarchical visual representation learning that assigns distinct learning objectives to global and interaction representations. | contribution 0.97 | 0.92 | 0.14 | asserted 1.00 |
| The global module learns object relations, task state, and temporal progress through future prediction on egocentric video. | method 0.99 | 0.79 | 0.06 | asserted 1.00 |
| The Interaction module interprets wrist observations in global context and learns local alignment and contact cues through robot action supervision. | method 0.99 | 0.75 | 0.06 | asserted 1.00 |
| Separate residual paths expose both representations to the action policy, preserving direct access to task context alongside fine-grained execution information. | method 0.99 | 0.83 | 0.07 | asserted 1.00 |
| Ego-CEVR supports independent pretraining of its global video module on large-scale human egocentric videos and videos from diverse robot embodiments, without requiring robot action labels or paired wrist observations. | method 0.58 | 0.83 | 0.16 | asserted 0.98 |
What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.53
What was examined, and what was not
Sources read
- arXiv listing abstract v1
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-28: 1 request, 5,592 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
Figures (3)
Figures are the authors', reproduced from the paper.
References (60)
- [1] Brianna Zitkovich, Tianhe Yu, Sichun Xu, Peng Xu, Ted Xiao, Fei Xia, Jialin Wu, Paul Wohlhart, Stefan Welker, Ayzaan Wahid, et al. RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control. In Conference on robot learning, pages 2165–2183. PMLR, 2023 on Pith
- [2] Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, et al. OpenVLA: An Open-Source Vision-Language-Action Model. arXiv preprint arXiv:2406.09246, 2024 on PithPith's reviewarXiv
- [3] Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, et al.π0: A Vision-Language-Action Flow Model for General Robot Control.arXiv preprint arXiv:2410.24164, 2024 on PithPith's reviewarXiv
- [4] Physical Intelligence, Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, et al. π0.5: a Vision-Language-Action Model with Open-World Generalization.arXiv preprint arXiv:2504.16054, 2025 on PithPith's reviewarXiv
- [5] Simar Kareer, Dhruv Patel, Ryan Punamiya, Pranay Mathur, Shuo Cheng, Chen Wang, Judy Hoffman, and Danfei Xu. EgoMimic: Scaling Imitation Learning via Egocentric Video. In2025 IEEE International Conference on Robotics and Automation (ICRA), pages 13226–13233. IEEE, 2025 on Pith
- [6] Cheng Chi, Zhenjia Xu, Chuer Pan, Eric Cousineau, Benjamin Burchfiel, Siyuan Feng, Russ Tedrake, and Shuran Song. Universal manipulation interface: In-the-wild robot teaching without in-the-wild robots.arXiv preprint arXiv:2402.10329, 2024 on PithPith's reviewarXiv
- [7] OX-Embodiment Collaboration, Abby O’Neill, Abdul Rehman, Abhinav Gupta, Abhiram Maddukuri, Abhishek Gupta, Abhishek Padalkar, Abraham Lee, Acorn Pooley, Agrim Gupta, et al. Open x-embodiment: Robotic learning datasets and rt-x models.arXiv preprint arXiv:2310.08864, 1(2), 2023 on PithPith's reviewarXiv
- [8] Yifan Ye, Yankai Fu, Yaoxu Lv, Bohan Hou, Jun Cen, Lingdong Kong, Duo Zheng, Tianxing Chen, Jiaming Liu, Ziang Cao, et al. Data pyramid for embodied manipulation: a survey.arXiv preprint arXiv:2607.24744, 2026 on PithPith's reviewarXiv
- [9] Fangcheng Liu, Yeqing Shen, Anda Cheng, Weishi Mi, Chao Tang, Chenyuan Liu, Yushun Xiang, Tingguang Li, Yong-Lu Li, and Yehui Tang. Roboicl: Embodied in-context learning with gpt-6 astra.arXiv preprint arXiv:2609.34261, 2026 on PithPith's reviewarXiv
- [10] Dongzhou Cheng, Taoran Yi, Ye Fang, Xingwu Zhang, Fan Feng, Yixuan Li, Gengxiong Zhuang, Rongze Wang, Shuai Yang, Wei Song, et al. In-context robot learning with vlm agents.arXiv preprint arXiv:2609.19138, 2026 on PithPith's reviewarXiv
- [11] Zhirui Fang, Qingchi Yu, Ziyang Chen, Longfei Li, Haoran Ma, Keru Zhou, Xinrun Xu, Samith Va, Yuxuan Hu, Peixuan Song, et al. Emerge-policy: A robot mind emerges beyond a single policy.arXiv preprint arXiv:2608.29896, 2026 on PithPith's reviewarXiv
- [12] Hung-Jen Chen, Yu-Hsun Hou, Yan-Hong Chen, Yan-Fu Chen, Binghua Cai, Min Sun, and Chun-Yi Lee. When instructions retrieve trajectories: Diagnosing and mitigating generalization failures in VLA models.arXiv preprint arXiv:2609.39971, 2026 on PithPith's reviewarXiv
- [13] Yu Fang, Yuchun Feng, Dong Jing, Jiaqi Liu, Yue Yang, Zhenyu Wei, Daniel Szafir, and Mingyu Ding. When vision overrides language: Evaluating and mitigating counterfactual failures in VLAs. Project website on Pith
- [14] Haochen Zhang and Yonatan Bisk. Flatness preserves instruction following in vision-language-action models. arXiv preprint arXiv:2606.23641, 2026 on PithPith's reviewarXiv
- [15] Yucheng Hu, Yanjiang Guo, Pengchao Wang, Xiaoyu Chen, Yen-Jen Wang, Jianke Zhang, Koushil Sreenath, Chaochao Lu, and Jianyu Chen. Video Prediction Policy: A Generalist Robot Policy with Predictive Visual Representations.arXiv preprint arXiv:2412.14803, 2024 on PithPith's reviewarXiv
- [16] Yue Liao, Pengfei Zhou, Siyuan Huang, Donglin Yang, Shengcong Chen, Yuxin Jiang, Yue Hu, Si Liu, Jianlan Luo, Liliang Chen, et al. Genie Envisioner: A Unified World Foundation Platform for Robotic Manipulation. In International Conference on Learning Representations, volume 20... on Pith
- [17] Renhang Liu, Wenzhi Zhao, Zhuo Yang, Liliang Chen, Pengfei Zhou, Shengcong Chen, Guanghui Ren, Youlun Peng, Rongjun Jin, Nan Wang, et al. GE-Act 2.0: Pretraining and Scaling a World-Action Model for Robotic Manipulation.arXiv preprint arXiv:2609.05588, 2026. 11 on PithPith's reviewarXiv
- [18] Shangchen Miao, Ningya Feng, Jialong Wu, Ye Lin, Xu He, Dong Li, and Mingsheng Long. JEPA-VLA: Video Predictive Embedding is Needed for VLA Models.arXiv preprint arXiv:2602.11832, 2026 on PitharXiv
- [19] Jiayi Chen, Wenxuan Song, Jingbo Wang, Shuai Zhou, Xicheng Gong, Zehua Fan, Ziyang Zhou, Haodong Yan, Fuhao Li, Qize Yu, et al. UniWAM: Unified World-Action Model.arXiv preprint arXiv:2610.02054, 2026 on PithPith's reviewarXiv
- [20] Simar Kareer, Karl Pertsch, James Darpinian, Judy Hoffman, Danfei Xu, Sergey Levine, Chelsea Finn, and Suraj Nair. Emergence of Human to Robot Transfer in Vision-Language-Action Models.arXiv preprint arXiv:2512.22414, 2025 on PitharXiv
show 40 more
- [21] Seonghyeon Ye, Yunhao Ge, Kaiyuan Zheng, Shenyuan Gao, Sihyun Yu, George Kurian, Suneel Indupuru, You Liang Tan, Chuning Zhu, Jiannan Xiang, et al. World Action Models are Zero-shot Policies.arXiv preprint arXiv:2602.15922, 2026 on PithPith's reviewarXiv
- [22] Zhihao Sun, Liu Liu, Xinjiang Wang, Haoyi Jiang, Wei Feng, Huiqiang Zhang, Xiaosong Jia, Zhizhong Su, and Zuxuan Wu. Ego4WAM: What Matters When Scaling Egocentric Human Data for Robot Learning?arXiv preprint arXiv:2609.40341, 2026 on PithPith's reviewarXiv
- [23] Bo Liu, Yifeng Zhu, Chongkai Gao, Yihao Feng, Qiang Liu, Yuke Zhu, and Peter Stone. LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning.Advances in Neural Information Processing Systems, 36:44776– 44791, 2023 on Pith
- [24] Senyu Fei, Siyin Wang, Junhao Shi, Zihao Dai, Jikun Cai, Pengfang Qian, Li Ji, Xinzhe He, Shiduo Zhang, Zhaoye Fei, et al. LIBERO-Plus: A Progressive Robustness Benchmark for Visual-Language-Action Models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patter... on Pith
- [25] Tianxing Chen, Zanxin Chen, Baijun Chen, Zijian Cai, Yibin Liu, Zixuan Li, Qiwei Liang, Xianliang Lin, Yiheng Ge, Zhenyu Gu, et al. RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation.arXiv preprint ar... on PithPith's reviewarXiv
- [26] Johan Bjorck, Fernando Castañeda, Nikita Cherniadev, Xingye Da, Runyu Ding, Linxi Fan, Yu Fang, Dieter Fox, Fengyuan Hu, Spencer Huang, et al. GR00T N1: An Open Foundation Model for Generalist Humanoid Robots. arXiv preprint arXiv:2503.14734, 2025 on PithPith's reviewarXiv
- [27] Gemini Robotics Team, Saminda Abeyruwan, Joshua Ainslie, Jean-Baptiste Alayrac, Montserrat Gonzalez Arenas, Travis Armstrong, Ashwin Balakrishna, Robert Baruch, Maria Bauza, Michiel Blokzijl, et al. Gemini Robotics: Bringing AI into the Physical World.arXiv preprint arXiv:2503... on PithPith's reviewarXiv
- [28] Hongtao Wu, Ya Jing, Chilam Cheang, Guangzeng Chen, Jiafeng Xu, Xinghang Li, Minghuan Liu, Hang Li, and Tao Kong. Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulation. InInternational Conference on Learning Representations, volume 2024, pages 1064... on Pith
- [29] Chi-Lam Cheang, Guangzeng Chen, Ya Jing, Tao Kong, Hang Li, Yifeng Li, Yuxiao Liu, Hongtao Wu, Jiafeng Xu, Yichu Yang, et al. GR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulation.arXiv preprint arXiv:2410.06158, 2024 on PithPith's reviewarXiv
- [30] Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, et al. Wan: Open and Advanced Large-Scale Video Generative Models.arXiv preprint arXiv:2503.20314, 2025 on PithPith's reviewarXiv
- [31] Niket Agarwal, Arslan Ali, Maciej Bala, Yogesh Balaji, Erik Barker, Tiffany Cai, Prithvijit Chattopadhyay, Yongxin Chen, Yin Cui, Yifan Ding, et al. Cosmos World Foundation Model Platform for Physical AI.arXiv preprint arXiv:2501.03575, 2025 on PithPith's reviewarXiv
- [32] Lin Li, Qihang Zhang, Yiming Luo, Shuai Yang, Ruilin Wang, Fei Han, Mingrui Yu, Zelin Gao, Nan Xue, Xing Zhu, et al. Causal World Modeling for Robot Control.arXiv preprint arXiv:2601.21998, 2026 on PithPith's reviewarXiv
- [33] Yanjiang Guo, Haodong Yan, Zhide Zhong, Zhongru Zhang, Qingyuan Yang, Qingzhou Lu, Xiaoyu Chen, Yen-Jen Wang, Shuying Deng, Chenghan Yang, et al. Video Prediction Policy 2: Predict Better, Act Better.arXiv preprint arXiv:2610.10270, 2026 on PithPith's reviewarXiv
- [34] Qihang Zhang, Lin Li, Luyao Zhang, Shuai Yang, Yiming Luo, Shuaiting Li, Ruilin Wang, Junke Wang, Jiahao Shao, Gangwei Xu, et al. Native Video-Action Pretraining for Generalizable Robot Control.arXiv preprint arXiv:2607.08639, 2026 on PithPith's reviewarXiv
- [35] Tianyuan Yuan, Zibin Dong, Yicheng Liu, and Hang Zhao. Fast-WAM: Do World Action Models Need Test-time Future Imagination?arXiv preprint arXiv:2603.16666, 2026. 12 on PithPith's reviewarXiv
- [36] Shuang Li, Yihuai Gao, Dorsa Sadigh, and Shuran Song. Unified Video Action Model.arXiv preprint arXiv:2503.00200, 2025 on PithPith's reviewarXiv
- [37] Chuning Zhu, Raymond Yu, Siyuan Feng, Benjamin Burchfiel, Paarth Shah, and Abhishek Gupta. Unified World Models: Coupling Video and Action Diffusion for Pretraining on Large Robotic Datasets.arXiv preprint arXiv:2504.02792, 2025 on PithPith's reviewarXiv
- [38] Hongzhe Bi, Hengkai Tan, Shenghao Xie, Zeyuan Wang, Shuhe Huang, Haitian Liu, Ruowen Zhao, Yao Feng, Chendong Xiang, Yinze Rong, et al. Motus: A Unified Latent Action World Model. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 35101–... on Pith
- [39] Ruijie Zheng, Jing Wang, Scott Reed, Johan Bjorck, Yu Fang, Fengyuan Hu, Joel Jang, Kaushil Kundalia, Zongyu Lin, Loic Magne, et al. FLARE: Robot Learning with Implicit World Modeling.arXiv preprint arXiv:2505.15659, 2025 on PithPith's reviewarXiv
- [40] Jingwen Sun, Wenyao Zhang, Zekun Qi, Shaojie Ren, Zezhi Liu, Hanxin Zhu, Guangzhong Sun, Xin Jin, and Zhibo Chen. VLA-JEPA: Enhancing Vision-Language-Action Model with Latent World Model. InEuropean Conference on Computer Vision, pages 478–497. Springer, 2026 on Pith
- [41] Suraj Nair, Aravind Rajeswaran, Vikash Kumar, Chelsea Finn, and Abhinav Gupta. R3M: A Universal Visual Representation for Robot Manipulation.arXiv preprint arXiv:2203.12601, 2022 on PithPith's reviewarXiv
- [42] Yecheng Jason Ma, Shagun Sodhani, Dinesh Jayaraman, Osbert Bastani, Vikash Kumar, and Amy Zhang. VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Training.arXiv preprint arXiv:2210.00030, 2022 on PithPith's reviewarXiv
- [43] Ruihan Yang, Qinxi Yu, Yecheng Wu, Rui Yan, Borui Li, An-Chieh Cheng, Xueyan Zou, Yunhao Fang, Xuxin Cheng, Ri-Zhao Qiu, et al. EgoVLA: Learning Vision-Language-Action Models from Egocentric Human Videos. arXiv preprint arXiv:2507.12440, 2025 on PithPith's reviewarXiv
- [44] Ruijie Zheng, Dantong Niu, Yuqi Xie, Jing Wang, Mengda Xu, Yunfan Jiang, Fernando Castañeda, Fengyuan Hu, You Liang Tan, Letian Fu, et al. EgoScale: Scaling Dexterous Manipulation with Diverse Egocentric Human Data.arXiv preprint arXiv:2602.16710, 2026 on PitharXiv
- [45] Baoyu Li, Xinchen Yin, Mengying Lin, Yixin Zhang, and Danfei Xu. EgoWAM: World Action Models Beyond Pixels with In-the-Wild Egocentric Human Data. InRobot World Models, 2026 on Pith
- [46] William Peebles and Saining Xie. Scalable Diffusion Models with Transformers. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 4195–4205, 2023 on Pith
- [47] Wan-Video. Wan2.2. Official code and model release, 2025. Released July 28, 2025; accessed October 22, 2026 on Pith
- [48] Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow Matching for Generative Modeling.arXiv preprint arXiv:2210.02747, 2022 on PithPith's reviewarXiv
- [49] Moo Jin Kim, Chelsea Finn, and Percy Liang. Fine-tuning vision-language-action models: Optimizing speed and success.arXiv preprint arXiv:2502.19645, 2025 on PithPith's reviewarXiv
- [50] Linqing Zhong, Yi Liu, Yifei Wei, Ziyu Xiong, Maoqing Yao, Si Liu, and Guanghui Ren. Acot-vla: Action chain-of-thought for vision-language-action models.arXiv preprint arXiv:2601.11404, 2026 on PitharXiv
- [51] Yihan Lin, Jiawei He, Shifeng Bao, Chen Zhao, Yang Li, Xiaobo Wang, Yan Wang, Cheng Chi, and Jing Zhang. Jepa-wam: Learning vision-language-action policies with joint-embedding world modeling.arXiv preprint arXiv:2608.09381, 2026 on PithPith's reviewarXiv
- [52] Ronghan Chen, Yandan Yang, Zuojin Tang, Dongjie Huo, Tong Lin, Haoning Wu, Haoyun Liu, Yuzhi Chen, Lulu Zheng, Botai Yuan, Tianlun Li, Mingxin Wang, Dekang Qi, Bin Hu, Wei Mei, Yuze Xuan, Haolong Yang, Yanqing Zhu, Mu Xu, Zhiheng Ma, and Xinyuan Chang. Abot-m0.5: Unified mobil... on PithPith's reviewarXiv
- [53] Hao Luo, Wanpeng Zhang, Yicheng Feng, Sipeng Zheng, Haiweng Xu, Chaoyi Xu, Ziheng Xi, Yuhui Fu, and Zongqing Lu. Being-H0.7: A Latent World-Action Model from Egocentric Videos.arXiv preprint arXiv:2605.00078, 2026. 13 on PithPith's reviewarXiv
- [54] Moo Jin Kim, Yihuai Gao, Tsung-Yi Lin, Yen-Chen Lin, Yunhao Ge, Grace Lam, Percy Liang, Shuran Song, Ming-Yu Liu, Chelsea Finn, et al. Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planning.arXiv preprint arXiv:2601.16163, 2026 on PithPith's reviewarXiv
- [55] Haisheng Su, Zongdai Liu, Xin Jin, Haoxuan Dou, Chengming Hu, Baorun Li, Zhanwang Liu, Ruiyan Xu, Jianjie Fang, Xin Zhang, Zhenjie Yang, Xue Yang, Chen Gao, Junchi Yan, Yong Li, and Wei Wu. Worldscape policy 2.0: Empowering steerable world action modeling with reasoning-augmen... on PithPith's reviewarXiv
- [56] Lishan Yang, Wenxuan Song, Xi Wang, Pingyue Sheng, Zheng Fang, Ziyang Zhou, Junjie He, Haodong Yan, Jiayi Chen, Nan Sun, Qiao Sun, Pengwei Wang, Lingqiao Liu, Yan Wang, Yuxiang Gao, Feras Dayoub, and Haoang Li. 4d-wam: Infusing spatiotemporal awareness into world action models... on PithPith's reviewarXiv
- [57] Jun Guo, Qiwei Li, Peiyan Li, Zilong Chen, Nan Sun, Yifei Su, Heyun Wang, Yuan Zhang, Xinghang Li, and Huaping Liu. Unified 4d world action modeling from video priors with asynchronous denoising.arXiv preprint arXiv:2604.26694, 2026 on PithPith's reviewarXiv
- [58] Yandan Yang, Shuang Zeng, Tong Lin, Xinyuan Chang, Dekang Qi, Junjin Xiao, Haoyun Liu, Ronghan Chen, Yuzhi Chen, Dongjie Huo, Feng Xiong, Xing Wei, Zhiheng Ma, and Mu Xu. Abot-m0: Vla foundation model for robotic manipulation with action manifold learning.arXiv preprint arXiv:... on PithPith's reviewarXiv
- [59] StarVLA Community. Starvla: A lego-like codebase for vision-language-action model developing.arXiv preprint arXiv:2604.05014, 2026 on PithPith's reviewarXiv
- [60] Galaxea Team. Galaxea g0: Open-world dataset and dual-system vla model.arXiv preprint arXiv:2509.00576, 2025. 14 on PithPith's reviewarXiv
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.
No comments yet.
Sign in to comment