Pith. sign in

REVIEW 5 major objections 4 minor 94 references

The paper claims that one video-model checkpoint, finetuned on about fifteen hours of masked clips, both simulates a robot's effect on a scene and recovers robot motion from desired object motion.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 12:41 UTC pith:LQBG4UTK

load-bearing objection Good idea, honest paper, but the mask-leakage confound and small application samples mean the claims need referee scrutiny before they can be taken at face value. the 5 major comments →

arxiv 2607.19343 v1 pith:LQBG4UTK submitted 2026-07-21 cs.CV cs.RO

Masked Visual Actions for Unified World Modeling

Hadi Alzayer , Wenlong Huang , Haonan Chen , Christopher Luey , Lvmin Zhang , Maneesh Agrawala , Gordon Wetzstein , Li Fei-Fei
show 3 more authors
Yilun Du Jiajun Wu Jia-Bin Huang
This is my paper
classification cs.CV cs.RO
keywords masked visual actionsvideo world modelsrobot manipulationforward dynamicsinverse dynamicspolicy evaluationmodel-based planningcross-embodiment generalization
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper is trying to establish that robot action, for a world model, can be expressed in the model's own visual space: as a partially revealed trajectory of pixels. Finetuning a single pre-trained video model on about fifteen hours of such masked examples produces one checkpoint that predicts the scene's response when the robot's motion is revealed and recovers robot motion when the desired object motion is revealed. If this works, a video model's pre-existing priors about motion and contact become usable as an action-conditioned simulator that transfers across embodiments without retraining. The same checkpoint is then used to rank candidate future actions in planning, evaluate policies by imagined rollouts, and synthesize executable robot behavior from a goal object trajectory. That is why a reader should care: a single pixel-level interface replaces embodiment-specific action encoders.

Core claim

The central discovery is that the forward and inverse problems of robot world modeling are two conditionings of one video-completion model. An entity's trajectory is revealed by a binary mask over its spatiotemporal pixel region; the model is trained to complete the masked video from a reference frame. When the revealed entity is the robot, the completion is the scene's reaction — a forward dynamics model. When the revealed entity is the manipulated object, the completion is the robot behavior consistent with that object motion — an inverse model. The same checkpoint, finetuned on roughly fifteen hours of real and simulated masked examples, does both, and because the conditioning lives in pi

What carries the argument

The central mechanism is a binary spatiotemporal mask M over pixel regions: for a chosen entity subset S, the input is the masked video M⊙V plus a reference image I0, and the model learns pθ(V | M⊙V, I0). Selecting active entities (the robot) yields p(passive | active, I0), the forward model; selecting passive entities (the object) yields p(active | passive, I0), the inverse model. The work this mask does is to keep the action signal pixel-aligned and embodiment-agnostic, so the recovered conditional distribution inherits the base video model's interaction priors rather than a task-specific action encoder. The partition into active and passive roles is an inference-time choice, not a trainin

Load-bearing premise

The masked pixels have to encode the robot's real motion: if segmentation or rendering masks are misaligned, occluded, or incomplete, the model is being trained on a corrupted action channel and the claimed forward/inverse behavior degrades.

What would settle it

On a held-out robot video with recorded joint states, render the true robot mask and a version of the same mask shifted a few frames along the recorded motion; generate completions from the same initial frame for both. If the two outputs are nearly identical or both match the ground-truth future, the model is not conditioning on the masked action, and the central claim fails.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Policy evaluation can be run partly in imagination: the model's simulated rollout success rates track ground-truth success rates closely enough to rank policies.
  • Model-based planning improves by sampling several candidate action sequences, simulating each, picking the best with an automated judge, and success rises with the number of candidates.
  • The same checkpoint, without task-specific inverse training, synthesizes robot motion from desired object motion and yields executable policies comparable to standard imitation learning baselines.
  • Because action conditioning is pixel-based rather than tied to joint or end-effector coordinates, a single model handles unseen robot embodiments, including morphologies with different numbers of arms.
  • Training on both successful and failed trajectories lets the model generate counterfactual outcomes, not only typical task completions.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the zero-shot forward-to-inverse transfer is general, pretrained video models may already contain much of the structure needed for action–outcome reasoning, which would reduce the amount of task-specific robot data required for new embodiments.
  • The active/passive querying idea extends beyond robots: any entity — a tool, a second agent, a human hand — could in principle be masked to ask questions about its motion, though the paper only tests robot and object roles.
  • The reported positive bias of simulated progress relative to real execution suggests that, as a policy evaluator, the model is optimistic; calibrating this bias per task could make its rankings more trustworthy than raw success rates.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper proposes Masked Visual Actions, a pixel-space conditioning interface for pretrained video models. Robot actions are communicated by revealing a masked spatiotemporal trajectory of an entity: revealing the robot makes the model act as a forward dynamics model, while revealing the manipulated object makes the same model act as an inverse model. The authors finetune Wan-Fun-Control 2.2 14B with LoRA on about 15 hours of masked examples from DROID and Robocasa, then evaluate the resulting single checkpoint on video reconstruction, model-based planning, policy evaluation, and action extraction. They report strong quantitative reconstruction results over Ctrl-World, Wan-Move, and image-to-video baselines, as well as generalization to unseen embodiments in BEHAVIOR-1K and a real-world setup. The paper also provides a detailed appendix with standard errors, VLM evaluation prompts, training details, and reproducibility commitments.

Significance. If the central claim holds, the paper offers an elegant unification: forward and inverse robot world modeling are cast as complementary conditional generation problems of a single video model, with a control interface that is embodiment-agnostic and directly aligned with the model's pretrained visual representation. The manuscript is transparent in several respects: Table C1 reports mean±SEM, the VLM judge prompt is included verbatim, the training data composition is specified, and code/weights are promised. The main risk is that the quantitative evidence rests on an unvalidated action channel and on several small-sample statistics; these need targeted experiments before the claims can be accepted as stated.

major comments (5)
  1. [§4.1; Tables 1 and C1] The action channel is never validated for accuracy or information leakage. Section 4.1 concedes that segmentation masks 'leak information about the scene dynamics from the original video' and that rendering requires camera calibration and can be misaligned, but no experiment measures mask IoU, pixel overlap with non-robot entities, or the effect of mask corruption on reconstruction metrics. If a SAM mask bleeds onto the manipulated object or background, the forward-model metrics in Tables 1 and C1 are inflated by copying revealed pixels; if an object mask overlaps the robot, the inverse-modeling result is similarly helped. Please add a leakage/misalignment study, including ground-truth mask comparisons on a subset and an ablation with deliberately corrupted or shifted masks.
  2. [§5.2, Fig. 9] The headline policy-evaluation correlation r=0.982 is computed over only 7 task-level points. With n=7, a single influential point can drive the correlation, and the 95% confidence interval is very wide. There is no nonparametric correlation, no interval, and no account of non-independence across rollouts from the same policy. Please report the full scatter with the CI, the Spearman correlation, and per-task standard errors. As written, 'exhibits a strong correlation' overstates the statistical evidence.
  3. [§5.2, Fig. 11; Action extraction] The action-extraction comparison uses 20 trials per method and reports 90% (ours) versus 85% (SmolVLA), 80% (ACT), and 50% (Diffusion Policy). The difference between 90% and 85% is within sampling noise for n=20; no error bars or significance tests are given, so 'achieves the highest success rate' is not supported. In addition, the reported number depends on both the video model and the separately trained inverse-dynamics model; please report variance over IDM training seeds and, ideally, direct metrics of synthesized robot-motion fidelity rather than only downstream task success.
  4. [§5.2, Fig. 10; Appendix F.3] The real-world policy-evaluation protocol is unclear and appears internally inconsistent. The main text and Fig. 10 describe four tasks (bagging orange, stacking blocks, placing towel, closing drawer) with 20 demonstrations each, while Appendix F.3 states that real-world policies were trained on only two tasks ('mug on rack' and 'close cabinet') with 40 trajectories. Please clarify whether Fig. 10 uses open-loop demonstrations or trained policies, list the exact task sets and trajectory counts, and explain the relationship between the 20-demonstration protocol and the 40-trajectory policy training described in the appendix.
  5. [§5.2, Fig. 8; Planning] The model-based planning results show per-task success-rate gains of 7–26 percentage points, but no error bars or significance tests are reported, and each task is evaluated on only 10 scenes. Because the VLM judge is also used to select rollouts, judge variability is unquantified. Please report per-scene SEM, paired tests across scenes, and the variance of the VLM selection step, for example by repeating the judge query or varying the judge temperature.
minor comments (4)
  1. [§4.2] Typo: 'F or reproducibility' should read 'For reproducibility.'
  2. [Appendix C] The sentence 'Our method consistently outperforms all the baselines, and within the standard error of conditioning on the skeleton for the real-world data' is ambiguous and partly contradicted by Table C1: for real-world PSNR, Ours (22.79±0.35) differs from Skeleton (21.02±0.38) by well over the combined standard error. Please rephrase and state explicitly which metric/condition the claim refers to.
  3. [Throughout] The benchmark name is typeset inconsistently as 'BEHA VIOR' and 'BEHAVIOR'; choose one form.
  4. [Table C1] The real-world column has n=13, but the main text does not describe how the 13 real-world scenes were selected or whether they overlap with the 20 demonstrations used in Fig. 10. Please add a sentence specifying the selection and the relationship.

Circularity Check

1 steps flagged

No meaningful circularity: the central forward/inverse claims are evaluated on held-out/external data; only a minor non-load-bearing self-citation (PointWorld calibration) and an acknowledged mask-leakage validity risk were found.

specific steps
  1. other [Section 4.1, Rendering based dataset; References [28]]
    "We use the DROID dataset and follow the protocols from PointWorld [28] to refine the camera calibration to accurately align the robot URDF with the input trajectories."

    PointWorld [28] is a prior paper whose first author (W. Huang) is also an author of this paper, making this a self-citation. It is used only as a camera-calibration protocol for the rendering-based data pipeline, not as a mathematical premise, uniqueness theorem, or source of the forward/inverse modeling claim. The central results are evaluated on held-out and external benchmarks (BEHAVIOR, Robocasa, real-world data), and the segmentation-based DROID pipeline provides an alternative data path. Hence this is a minor, non-load-bearing self-citation rather than a demonstration of circular derivation.

full rationale

The claimed derivation chain is Eqs. (1)-(5): a pretrained video model is finetuned on masked video completion, and forward/inverse behavior is obtained by choosing whether the revealed mask is the active entity (robot) or the passive entity (object). Nothing in this chain is defined in terms of the target result, and no fitted parameter is later renamed as a prediction. Reconstruction metrics are computed on held-out DROID scenes, unseen BEHAVIOR embodiments, and self-captured real-world data; planning and policy evaluation use Robocasa actions not directly fit to the reported success rates; action extraction is tested on COFFEESERVEMUG, which the paper states the video model has not seen, with success measured by actual execution rather than by re-fitting. The only self-citation entering the method is the PointWorld calibration protocol [28], which is a data-processing recipe whose first author overlaps with this paper; it is not load-bearing for the central claims, so it is a minor concern rather than circularity. The paper itself notes in Section 4.1 that segmentation-based masks can 'implicitly leak information about the scene dynamics from the original video,' and it does not state whether the reported reconstruction metrics exclude the revealed entity pixels; these are evaluation-validity risks for the DROID pixel metrics, not demonstrated circular identities, and they do not affect the downstream, externally scored applications. Overall, the derivation is self-contained and independently benchmarked; no significant circularity is present.

Axiom & Free-Parameter Ledger

8 free parameters · 6 axioms · 0 invented entities

The paper introduces a new control interface (masked visual actions), not a new physical entity. The central claim rests on several hyperparameters, the base video model's prior, and data pipelines (segmentation/rendering) that are not yet released.

free parameters (8)
  • LoRA rank = 256
    Chosen hyperparameter; scales capacity of adaptation and affects fidelity.
  • Training steps = ~10,000 (4 days on 8 H200 GPUs)
    Chosen by compute budget; performance may vary with more/less training.
  • Training data composition = 1,000 DROID demos + 4,000 Robocasa examples
    Chosen data mix; domain coverage and balance affect generalization.
  • Mask rendering style = translucent robot with bright red gripper fingers
    Chosen visual design; affects learnability of the masked action signal.
  • Mask distribution = not specified
    Section 3 says 'sampling M from a distribution over masks' but never specifies the distribution; this is a key detail for training and reproducibility.
  • VLM judge = Gemini 3.1 Pro Preview with hand-crafted rubric and lexicographic ordering κ
    Planning results and evaluation depend on this proprietary model and the author-defined rubric; changing it could change conclusions.
  • Planning sample count N = 10 rollouts per scene
    Chosen evaluation setup; success-rate curves use subset enumeration of these samples.
  • IDM training epochs = 300
    Hyperparameter for the inverse-dynamics model used in action extraction (Appendix F.2).
axioms (6)
  • domain assumption Base video model Wan-Fun-Control 2.2 14B has learned broad interaction priors from pretraining
    Central premise: the method relies on a strong pretrained video prior being steerable by masked trajectories. If the base model lacks physical interaction priors, the finetuning cannot produce realistic rollouts.
  • domain assumption SAM segmentation with prompt 'A robotic arm' reliably identifies robot pixels in DROID videos
    Section 4.1 uses SAM to generate masks. Segmentation failures or missed robot regions would corrupt the conditioning signal.
  • domain assumption URDF rendering + refined camera calibration produces masks aligned with the scene
    Section 4.1 requires accurate camera calibration and robot state. Misalignment injects noise into the action channel.
  • domain assumption The VLM judge can accurately score task success, contact, and physical realism from videos
    Planning and policy-evaluation protocols rely on Gemini 3.1 Pro Preview to judge rollouts; if the judge is unreliable, the reported planning gains and evaluation correlations may not reflect true physical validity.
  • domain assumption DROID/Robocasa trajectory coverage is sufficient for finetuning a generalizable masked-action model
    The model is finetuned on ~15h of interaction data; the paper assumes this narrow distribution transfers to unseen embodiments and tasks.
  • domain assumption The video model's learned joint distribution factorizes over entities as p(e1,...,en)
    Section 3 models the video as a joint distribution over entity trajectories; this is a modeling assumption about the expressivity of the video model and the separability of entities.

pith-pipeline@v1.3.0-alltime-deepseek · 19650 in / 13769 out tokens · 140050 ms · 2026-08-01T12:41:52.257914+00:00 · methodology

0 comments
read the original abstract

Video models absorb rich priors over how the visual world moves, interacts, and responds to contact, making them promising substrates for robotic world modeling. The central challenge is how to communicate action to such models in a form aligned with the visual space in which they learned these interaction priors, yet still grounded in physical manipulation. We introduce Masked Visual Actions, a pixel-space control interface that expresses action as a partially revealed trajectory of an arbitrary entity in a video. Revealing robot motion makes the model act as a forward dynamics model that predicts the scene's response to low-level robot actions, while revealing desired object motion makes the same model recover robot behavior consistent with that outcome. Finetuned with only 15 hours of masked examples from real videos and simulation, a single checkpoint achieves strong visual fidelity and controllability across diverse scenes and multiple embodiments. In downstream manipulation settings, the model produces imagined rollouts whose outcomes correlate with real-world execution for policy evaluation, improves decision making by ranking candidate futures in model-based planning, and supports inverse modeling by synthesizing robot motion from desired object motion.

Figures

Figures reproduced from arXiv: 2607.19343 by Christopher Luey, Gordon Wetzstein, Hadi AlZayer, Haonan Chen, Jia-Bin Huang, Jiajun Wu, Li Fei-Fei, Lvmin Zhang, Maneesh Agrawala, Wenlong Huang, Yilun Du.

Figure 1
Figure 1. Figure 1: Masked Visual Actions. We finetune a video model to condition on masked trajectories of robots, representing robot actions as pixel-space masked motions. Efficiently finetuned on only 15 hours of data, a single checkpoint of the model can act as an action-conditioned forward model to simulate robotic interactions with diverse and unseen embodiments. By conditioning it on object motion, it can also act as a… view at source ↗
Figure 2
Figure 2. Figure 2: Comparing action representations for learning. Low-dimensional robot actions are compact, but embodiment-specific and not aligned with the image observations used by video models. End-effector poses or robot skeletons are more visual, but remain sparse and require the model to infer geometry, contact, and interaction effects. Our masked visual actions provide dense, image-aligned conditioning, making robot… view at source ↗
Figure 3
Figure 3. Figure 3: Applications. The masked visual actions allow using the video model as a forward model, conditioned on robot actions, or as an inverse model that predicts the robot motion that satisfies the object trajectory. The forward model can be used for planning and choosing the best trajectory sampled from a policy, or policy evaluation. On the other hand, the inverse modeling can be combined with an inverse dynami… view at source ↗
Figure 4
Figure 4. Figure 4: Dataset construction. (a) to train our model, we need a reference frame of the initial scene, and the masked visual actions, and train it to reproduce a realistic video of the robot executing the input actions. (b) We use segmentation-based approach by segmenting the robot arm from robotics datasets as the masked visual actions. (c) However, to allow the user to provide arbitrary action trajectory at infer… view at source ↗
Figure 6
Figure 6. Figure 6: Comparing baselines on DROID. Using image-to-video [63] or even trajectory conditioned video generation [13] with GT tracks fails to execute the robot motion or preserve the input scene. On the other hand, our model can competitively match and outperform models that take the raw robot actions [25] while maintaining generalization [PITH_FULL_IMAGE:figures/full_fig_p006_6.png] view at source ↗
Figure 5
Figure 5. Figure 5: Generalization to unseen embodiment. While using the raw action state such as in Ctrl￾world [25] can work well within the training domain, it collapses on unseen embodiments. However, our method can generalize well to unseen embodiments. We start by evaluating Masked Visual Actions as a control signal for world modeling. We eval￾uate visual fidelity and controllability against prior work and highlight gene… view at source ↗
Figure 7
Figure 7. Figure 7: Comparing action conditioning. Training a video model on different conditioning signals on DROID such as masked visual actions, end effector visualization, or skeleton all work well within the training domain. However, when going beyond the training distribution, such as using a custom end-effector, the models trained on skeleton and end effector position would hallucinate the robot seen in training or tra… view at source ↗
Figure 8
Figure 8. Figure 8: Application on Planning. By rolling out multiple trajectories from a pretrained diffusion policy, we can evaluate each trajectory by simulating the actions with the video model, and then using a VLM judge to pick the best action trajectory. We observe consistent improvement in task success when using the video model to roll out and choose best action sequences, as well as the positive correlation with the … view at source ↗
Figure 9
Figure 9. Figure 9: Robocasa policy evaluation. Video-model rollouts consistently track ground-truth success rates across RoboCasa tasks. bagging orange stacking blocks placing towel closing drawer 0% 25% 50% 75% 100% Per-trial progress Ground Truth Sim Mean [PITH_FULL_IMAGE:figures/full_fig_p009_9.png] view at source ↗
Figure 11
Figure 11. Figure 11: Action extraction. Even without task-specific video-model training, inverse modeling recovers competitive robot behavior. However, we observe that the video model shows a positive bias towards task progress, as evidenced by consistently higher task success rates in its imagination. Beyond simulation, we evaluate our model in a real-world setup. For each of four tasks we collect 20 demonstrations, roll out… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

94 extracted references · 1 canonical work pages

  1. [1]

    Amir Bar, Yossi Gandelsman, Trevor Darrell, Amir Globerson, and Alexei A. Efros. Visual prompting via image inpainting. InNeurIPS, 2022

    2022

  2. [2]

    Unifying (machine) vision via counterfactual world modeling.arXiv preprint arXiv:2306.01828, 2023

    Daniel M Bear, Kevin Feigelis, Honglin Chen, Wanhee Lee, Rahul Venkatesh, Klemen Kotar, Alex Durango, and Daniel LK Yamins. Unifying (machine) vision via counterfactual world modeling.arXiv preprint arXiv:2306.01828, 2023

    Pith/arXiv arXiv 2023

  3. [3]

    Track2act: Predicting point tracks from internet videos enables generalizable robot manipulation, 2024

    Homanga Bharadhwaj, Roozbeh Mottaghi, Abhinav Gupta, and Shubham Tulsiani. Track2act: Predicting point tracks from internet videos enables generalizable robot manipulation, 2024

    2024

  4. [4]

    Video generation models as world simulators

    Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, Clarence Ng, Ricky Wang, and Aditya Ramesh. Video generation models as world simulators. 2024. URL https://openai.com/research/ video-generation-models-as-world-simulators

    2024

  5. [5]

    Go-with-the-flow: Motion-controllable video diffusion models using real-time warped noise

    Ryan Burgert, Yuancheng Xu, Wenqi Xian, Oliver Pilarski, Pascal Clausen, Mingming He, Li Ma, Yitong Deng, Lingxiao Li, Mohsen Mousavi, Michael Ryoo, Paul Debevec, and Ning Yu. Go-with-the-flow: Motion-controllable video diffusion models using real-time warped noise. In CVPR, 2025

    2025

  6. [6]

    SAM 3: Segment anything with concepts

    Nicolas Carion, Laura Gustafson, Yuan-Ting Hu, Shoubhik Debnath, Ronghang Hu, Didac Suris Coll-Vinent, Chaitanya Ryali, Kalyan Vasudev Alwala, Haitham Khedr, Andrew Huang, Jie Lei, Tengyu Ma, Baishan Guo, Arpit Kalla, Markus Marks, Joseph Greer, Meng Wang, Peize Sun, Roman Rädle, Triantafyllos Afouras, Effrosyni Mavroudi, Katherine Xu, Tsung-Han Wu, Yu Zh...

    2026

  7. [7]

    Freeman, Jitendra Malik, Russ Tedrake, Vincent Sitzmann, and Yilun Du

    Boyuan Chen, Tianyuan Zhang, Haoran Geng, Kiwhan Song, William T. Freeman, Jitendra Malik, Russ Tedrake, Vincent Sitzmann, and Yilun Du. Large video planner, 2025

    2025

  8. [8]

    Learning coordinated bimanual manipulation policies using state diffusion and inverse dynamics models

    Haonan Chen, Jiaming Xu, Lily Sheng, Tianchen Ji, Shuijing Liu, Yunzhu Li, and Katherine Driggs-Campbell. Learning coordinated bimanual manipulation policies using state diffusion and inverse dynamics models. In2025 IEEE International Conference on Robotics and Automation (ICRA), 2025. 10

    2025

  9. [9]

    Tool-as-interface: Learning robot policies from observing human tool use

    Haonan Chen, Cheng Zhu, Shuijing Liu, Yunzhu Li, and Katherine Rose Driggs-Campbell. Tool-as-interface: Learning robot policies from observing human tool use. InProceedings of Robotics: Conference on Robot Learning (CoRL), 2025

    2025

  10. [10]

    Bridgev2w: Bridging video generation models to embodied world models via embodiment masks.arXiv preprint arXiv:2602.03793, 2026

    Yixiang Chen, Peiyan Li, Jiabing Yang, Keji He, Xiangnan Wu, Yuan Xu, Kai Wang, Jing Liu, Nianfeng Liu, Yan Huang, et al. Bridgev2w: Bridging video generation models to embodied world models via embodiment masks.arXiv preprint arXiv:2602.03793, 2026

    arXiv 2026

  11. [11]

    Diffusion policy: Visuomotor policy learning via action diffusion

    Cheng Chi, Siyuan Feng, Yilun Du, Zhenjia Xu, Eric Cousineau, Benjamin Burchfiel, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. InProceedings of Robotics: Science and Systems (RSS), 2023

    2023

  12. [12]

    Diffusion policy: Visuomotor policy learning via action diffusion

    Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. The International Journal of Robotics Research, 2024

    2024

  13. [13]

    Wan-move: Motion-controllable video generation via latent trajectory guidance.arXiv preprint arXiv:2512.08765, 2025

    Ruihang Chu, Yefei He, Zhekai Chen, Shiwei Zhang, Xiaogang Xu, Bin Xia, Dingdong Wang, Hongwei Yi, Xihui Liu, Hengshuang Zhao, et al. Wan-move: Motion-controllable video generation via latent trajectory guidance.arXiv preprint arXiv:2512.08765, 2025

    arXiv 2025

  14. [14]

    Embodis- wap for zero-shot robot imitation learning, 2025

    Eadom Dessalene, Pavan Mantripragada, Michael Maynord, and Yiannis Aloimonos. Embodis- wap for zero-shot robot imitation learning, 2025

    2025

  15. [15]

    BERT: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Jill Burstein, Christy Doran, and Thamar Solorio, editors,Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, V ol...

    doi:10.18653/v1/ 2019

  16. [16]

    Learning universal policies via text-guided video generation.Advances in neural information processing systems, 36:9156–9172, 2023

    Yilun Du, Sherry Yang, Bo Dai, Hanjun Dai, Ofir Nachum, Josh Tenenbaum, Dale Schuurmans, and Pieter Abbeel. Learning universal policies via text-guided video generation.Advances in neural information processing systems, 36:9156–9172, 2023

    2023

  17. [17]

    Video language planning

    Yilun Du, Sherry Yang, Pete Florence, Fei Xia, Ayzaan Wahid, Pierre Sermanet, Tianhe Yu, Pieter Abbeel, Joshua B Tenenbaum, Leslie Pack Kaelbling, et al. Video language planning. In The Twelfth International Conference on Learning Representations, 2023

    2023

  18. [18]

    Aim: Intent-aware unified world action modeling with spatial value maps, 2026

    Liaoyuan Fan, Zetian Xu, Chen Cao, Wenyao Zhang, Mingqi Yuan, and Jiayu Chen. Aim: Intent-aware unified world action modeling with spatial value maps, 2026

    2026

  19. [19]

    Zentner, George Kurian, Suneel Indupuru, Pooya Jannaty, Jinwei Gu, Jun Zhang, Jitendra Malik, Pieter Abbeel, Ming-Yu Liu, Yuke Zhu, Joel Jang, and Linxi "Jim" Fan

    Shenyuan Gao, William Liang, Kaiyuan Zheng, Ayaan Malik, Seonghyeon Ye, Sihyun Yu, Wei-Cheng Tseng, Yuzhu Dong, Kaichun Mo, Chen-Hsuan Lin, Qianli Ma, Seungjun Nah, Loic Magne, Jiannan Xiang, Yuqi Xie, Ruijie Zheng, Dantong Niu, You Liang Tan, K.R. Zentner, George Kurian, Suneel Indupuru, Pooya Jannaty, Jinwei Gu, Jun Zhang, Jitendra Malik, Pieter Abbeel,...

    Pith/arXiv arXiv 2026

  20. [20]

    Motion prompting: Controlling video generation with motion trajectories

    Daniel Geng, Charles Herrmann, Junhwa Hur, Forrester Cole, Serena Zhang, Tobias Pfaff, Tatiana Lopez-Guevara, Carl Doersch, Yusuf Aytar, Michael Rubinstein, Chen Sun, Oliver Wang, Andrew Owens, and Deqing Sun. Motion prompting: Controlling video generation with motion trajectories. InCVPR, 2025

    2025

  21. [21]

    Force prompting: Video generation models can learn and generalize physics-based control signals

    Nate Gillman, Charles Herrmann, Michael Freeman, Daksh Aggarwal, Evan Luo, Deqing Sun, and Chen Sun. Force prompting: Video generation models can learn and generalize physics-based control signals. InNeurIPS, 2025

    2025

  22. [22]

    Goal force: Teaching video models to accomplish physics-conditioned goals

    Nate Gillman, Yinghua Zhou, Zitian Tang, Evan Luo, Arjan Chakravarthy, Daksh Aggarwal, Michael Freeman, Charles Herrmann, and Chen Sun. Goal force: Teaching video models to accomplish physics-conditioned goals. InCVPR, 2026

    2026

  23. [23]

    World models for learning dexterous hand-object interactions from human videos.arXiv preprint arXiv:2512.13644, 2026

    Raktim Gautam Goswami, Amir Bar, David Fan, Tsung-Yen Yang, Gaoyue Zhou, Prashanth Kr- ishnamurthy, Michael Rabbat, Farshad Khorrami, and Yann LeCun. World models for learning dexterous hand-object interactions from human videos.arXiv preprint arXiv:2512.13644, 2026. 11

    arXiv 2026

  24. [24]

    Unified 4d world action modeling from video priors with asynchronous denoising, 2026

    Jun Guo, Qiwei Li, Peiyan Li, Zilong Chen, Nan Sun, Yifei Su, Heyun Wang, Yuan Zhang, Xinghang Li, and Huaping Liu. Unified 4d world action modeling from video priors with asynchronous denoising, 2026

    2026

  25. [25]

    Ctrl-world: A controllable generative world model for robot manipulation

    Yanjiang Guo, Lucy Xiaoyang Shi, Jianyu Chen, and Chelsea Finn. Ctrl-world: A controllable generative world model for robot manipulation. InICLR, 2026

    2026

  26. [26]

    Video prediction policy: A generalist robot policy with predictive visual representations, 2024

    Yucheng Hu, Yanjiang Guo, Pengchao Wang, Xiaoyu Chen, Yen-Jen Wang, Jianke Zhang, Koushil Sreenath, Chaochao Lu, and Jianyu Chen. Video prediction policy: A generalist robot policy with predictive visual representations, 2024

    2024

  27. [27]

    Vid2world: Crafting video diffusion models to interactive world models, 2025

    Siqiao Huang, Jialong Wu, Qixing Zhou, Shangchen Miao, and Mingsheng Long. Vid2world: Crafting video diffusion models to interactive world models, 2025

    2025

  28. [28]

    Pointworld: Scaling 3d world models for in-the-wild robotic manipulation

    Wenlong Huang, Yu-Wei Chao, Arsalan Mousavian, Ming-Yu Liu, Dieter Fox, Kaichun Mo, and Fei-Fei Li. Pointworld: Scaling 3d world models for in-the-wild robotic manipulation. In CVPR, 2026

    2026

  29. [29]

    Dreamgen: Unlocking generalization in robot learning through video world models, 2025

    Joel Jang, Seonghyeon Ye, Zongyu Lin, Jiannan Xiang, Johan Bjorck, Yu Fang, Fengyuan Hu, Spencer Huang, Kaushil Kundalia, Yen-Chen Lin, Loic Magne, Ajay Mandlekar, Avnish Narayan, You Liang Tan, Guanzhi Wang, Jing Wang, Qi Wang, Yinzhen Xu, Xiaohui Zeng, Kaiyuan Zheng, Ruijie Zheng, Ming-Yu Liu, Luke Zettlemoyer, Dieter Fox, Jan Kautz, Scott Reed, Yuke Zh...

    2025

  30. [30]

    Cotracker3: Simpler and better point tracking by pseudo-labelling real videos, 2024

    Nikita Karaev, Iurii Makarov, Jianyuan Wang, Natalia Neverova, Andrea Vedaldi, and Christian Rupprecht. Cotracker3: Simpler and better point tracking by pseudo-labelling real videos, 2024

    2024

  31. [31]

    Alexander Khazatsky, Karl Pertsch, Suraj Nair, Ashwin Balakrishna, Sudeep Dasari, Siddharth Karamcheti, Soroush Nasiriany, Mohan Kumar Srirama, Lawrence Yunliang Chen, Kirsty Ellis, Peter David Fagan, Joey Hejna, Masha Itkina, Marion Lepert, Yecheng Jason Ma, Patrick Tree Miller, Jimmy Wu, Suneel Belkhale, Shivin Dass, Huy Ha, Arhan Jain, Abraham Lee, You...

    2024

  32. [32]

    Dexterous world models

    Byungjun Kim, Taeksoo Kim, Junyoung Lee, and Hanbyul Joo. Dexterous world models. In CVPR, 2026

    2026

  33. [33]

    Cosmos policy: Fine-tuning video models for visuomotor control and planning, 2026

    Moo Jin Kim, Yihuai Gao, Tsung-Yi Lin, Yen-Chen Lin, Yunhao Ge, Grace Lam, Percy Liang, Shuran Song, Ming-Yu Liu, Chelsea Finn, and Jinwei Gu. Cosmos policy: Fine-tuning video models for visuomotor control and planning, 2026

    2026

  34. [34]

    Learning to act from actionless videos through dense correspondences.arXiv preprint arXiv:2310.08576, 2023

    Po-Chen Ko, Jiayuan Mao, Yilun Du, Shao-Hua Sun, and Joshua B Tenenbaum. Learning to act from actionless videos through dense correspondences.arXiv preprint arXiv:2310.08576, 2023

    Pith/arXiv arXiv 2023

  35. [35]

    World modeling with probabilistic structure integration.arXiv preprint arXiv:2509.09737, 2025

    Klemen Kotar, Wanhee Lee, Rahul Venkatesh, Honglin Chen, Daniel Bear, Jared Watrous, Simon Kim, Khai Loong Aw, Lilian Naing Chen, Stefan Stojanov, et al. World modeling with probabilistic structure integration.arXiv preprint arXiv:2509.09737, 2025

    Pith/arXiv arXiv 2025

  36. [36]

    Shadow: Leveraging segmentation masks for cross-embodiment policy transfer, 2025

    Marion Lepert, Ria Doshi, and Jeannette Bohg. Shadow: Leveraging segmentation masks for cross-embodiment policy transfer, 2025. 12

    2025

  37. [37]

    Masquerade: Learning from in-the-wild human videos using data-editing, 2025

    Marion Lepert, Jiaying Fang, and Jeannette Bohg. Masquerade: Learning from in-the-wild human videos using data-editing, 2025

    2025

  38. [38]

    Phantom: Training robots without robots using only human videos, 2025

    Marion Lepert, Jiaying Fang, and Jeannette Bohg. Phantom: Training robots without robots using only human videos, 2025

    2025

  39. [39]

    Karen Liu, Jiajun Wu, and Li Fei-Fei

    Chengshu Li, Ruohan Zhang, Josiah Wong, Cem Gokmen, Sanjana Srivastava, Roberto Martín- Martín, Chen Wang, Gabrael Levine, Wensi Ai, Benjamin Martinez, Hang Yin, Michael Lingelbach, Minjune Hwang, Ayano Hiranaka, Sujay Garlanka, Arman Aydin, Sharon Lee, Jiankai Sun, Mona Anvari, Manasi Sharma, Dhruva Bansal, Samuel Hunter, Kyu-Young Kim, Alan Lou, Caleb R...

    Pith/arXiv arXiv 2024

  40. [40]

    Mask2iv: Interaction-centric video generation via mask trajectories, 2025

    Gen Li, Bo Zhao, Jianfei Yang, and Laura Sevilla-Lara. Mask2iv: Interaction-centric video generation via mask trajectories, 2025

    2025

  41. [41]

    Novaflow: Zero-shot manipulation via actionable flow from generated videos, 2025

    Hongyu Li, Lingfeng Sun, Yafei Hu, Duy Ta, Jennifer Barry, George Konidaris, and Jiahui Fu. Novaflow: Zero-shot manipulation via actionable flow from generated videos, 2025

    2025

  42. [42]

    Unified video action model, 2025

    Shuang Li, Yihuai Gao, Dorsa Sadigh, and Shuran Song. Unified video action model, 2025

    2025

  43. [43]

    Genie envisioner: A unified world foundation platform for robotic manipulation, 2025

    Yue Liao, Pengfei Zhou, Siyuan Huang, Donglin Yang, Shengcong Chen, Yuxin Jiang, Yue Hu, Jingbin Cai, Si Liu, Jianlan Luo, Liliang Chen, Shuicheng Yan, Maoqing Yao, and Guanghui Ren. Genie envisioner: A unified world foundation platform for robotic manipulation, 2025

    2025

  44. [44]

    Realwonder: Real-time physical action-conditioned video generation, 2026

    Wei Liu, Ziyu Chen, Zizhang Li, Yue Wang, Hong-Xing Yu, and Jiajun Wu. Realwonder: Real-time physical action-conditioned video generation, 2026

    2026

  45. [45]

    Zero-shot world models are developmentally efficient learners.arXiv e-prints, pages arXiv–2604, 2026

    Khai Loong Aw, Klemen Kotar, Wanhee Lee, Seungwoo Kim, Khaled Jedoui, Rahul Venkatesh, Lilian Naing Chen, Michael C Frank, and Daniel LK Yamins. Zero-shot world models are developmentally efficient learners.arXiv e-prints, pages arXiv–2604, 2026

    2026

  46. [46]

    Fixing weight decay regularization in adam.arXiv preprint arXiv:1711.05101, 5(5):5, 2017

    Ilya Loshchilov, Frank Hutter, et al. Fixing weight decay regularization in adam.arXiv preprint arXiv:1711.05101, 5(5):5, 2017

    Pith/arXiv arXiv 2017

  47. [47]

    Mask world model: Predicting what matters for robust robot policy learning, 2026

    Yunfan Lou, Xiaowei Chi, Xiaojie Zhang, Zezhong Qian, Chengxuan Li, Rongyu Zhang, Yaoxu Lyu, Guoyu Song, Chuyao Fu, Haoxuan Xu, Pengwei Wang, and Shanghang Zhang. Mask world model: Predicting what matters for robust robot policy learning, 2026

    2026

  48. [48]

    Inference-time scaling for diffusion models beyond scaling denoising steps

    Nanye Ma, Shangyuan Tong, Haolin Jia, Hexiang Hu, Yu-Chuan Su, Mingda Zhang, Xuan Yang, Yandong Li, Tommi Jaakkola, Xuhui Jia, and Saining Xie. Inference-time scaling for diffusion models beyond scaling denoising steps. 2025

    2025

  49. [49]

    Mimicgen: A data generation system for scalable robot learning using human demonstrations.arXiv preprint arXiv:2310.17596, 2023

    Ajay Mandlekar, Soroush Nasiriany, Bowen Wen, Iretiayo Akinola, Yashraj Narang, Linxi Fan, Yuke Zhu, and Dieter Fox. Mimicgen: A data generation system for scalable robot learning using human demonstrations.arXiv preprint arXiv:2310.17596, 2023

    Pith/arXiv arXiv 2023

  50. [50]

    Motubrain: An advanced world action model for robot control, 2026

    MotuBrain Team, Chendong Xiang, Fan Bao, Haitian Liu, Hengkai Tan, Hongzhe Bi, James Li, Jiabao Liu, Jingrui Pang, Kiro Jing, Louis Liu, Mengchen Cai, Rongxu Cui, Ruowen Zhao, Runqing Wang, Shuhe Huang, Yao Feng, Yinze Rong, Zeyuan Wang, and Jun Zhu. Motubrain: An advanced world action model for robot control, 2026

    2026

  51. [51]

    s1: Simple test-time scaling

    Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. s1: Simple test-time scaling. In Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng, editors,Proceedings of the 2025 Conference on Empirical Methods in Natural Lan...

    2025

  52. [52]

    Robocasa365: A large-scale simulation framework for training and benchmarking generalist robots

    Soroush Nasiriany, Sepehr Nasiriany, Abhiram Maddukuri, and Yuke Zhu. Robocasa365: A large-scale simulation framework for training and benchmarking generalist robots. InICLR, 2026

    2026

  53. [53]

    Cosmos world foundation model platform for physical ai, 2025

    NVIDIA, :, Niket Agarwal, Arslan Ali, Maciej Bala, Yogesh Balaji, Erik Barker, Tiffany Cai, Prithvijit Chattopadhyay, Yongxin Chen, Yin Cui, Yifan Ding, Daniel Dworakowski, Jiaojiao Fan, Michele Fenzi, Francesco Ferroni, Sanja Fidler, Dieter Fox, Songwei Ge, Yunhao Ge, Jinwei Gu, Siddharth Gururani, Ethan He, Jiahui Huang, Jacob Huffman, Pooya Jannaty, Ji...

    2025

  54. [54]

    mimic-video: Video-action models for generalizable robot control beyond vlas, 2025

    Jonas Pai, Liam Achenbach, Victoriano Montesinos, Benedek Forrai, Oier Mees, and Elvis Nava. mimic-video: Video-action models for generalizable robot control beyond vlas, 2025

    2025

  55. [55]

    Inference-time enhancement of generative robot policies via predictive world modeling.IEEE Robotics and Automation Letters, 2026

    Han Qi, Haocheng Yin, Aris Zhu, Yilun Du, and Heng Yang. Inference-time enhancement of generative robot policies via predictive world modeling.IEEE Robotics and Automation Letters, 2026

    2026

  56. [56]

    MotionStream: Real-Time Video Generation with Interactive Motion Controls

    Joonghyuk Shin, Zhengqi Li, Richard Zhang, Jun-Yan Zhu, Jaesik Park, Eli Shechtman, and Xun Huang. MotionStream: Real-Time Video Generation with Interactive Motion Controls. In ICLR, 2026

    2026

  57. [57]

    Smolvla: A vision-language-action model for affordable and efficient robotics.arXiv preprint arXiv:2506.01844, 2025

    Mustafa Shukor, Dana Aubakirova, Francesco Capuano, Pepijn Kooijmans, Steven Palma, Adil Zouitine, Michel Aractingi, Caroline Pascal, Martino Russi, Andres Marafioti, et al. Smolvla: A vision-language-action model for affordable and efficient robotics.arXiv preprint arXiv:2506.01844, 2025

    Pith/arXiv arXiv 2025

  58. [58]

    Time-to-move: Training-free motion-controlled video generation via dual-clock denoising

    Assaf Singer, Noam Rotstein, Amir Mann, Ron Kimmel, and Or Litany. Time-to-move: Training-free motion-controlled video generation via dual-clock denoising. InICLR, 2026

    2026

  59. [59]

    Motion before action: Diffusing object motion as manipulation condition, 2024

    Yue Su, Xinyu Zhan, Hongjie Fang, Yong-Lu Li, Cewu Lu, and Lixin Yang. Motion before action: Diffusing object motion as manipulation condition, 2024

    2024

  60. [60]

    Evaluating gemini robotics policies in a veo world simulator, 2025

    Gemini Robotics Team, Krzysztof Choromanski, Coline Devin, Yilun Du, Debidatta Dwibedi, Ruiqi Gao, Abhishek Jindal, Thomas Kipf, Sean Kirmani, Isabel Leal, Fangchen Liu, Anirudha Majumdar, Andrew Marmon, Carolina Parada, Yulia Rubanova, Dhruv Shah, Vikas Sindhwani, Jie Tan, Fei Xia, Ted Xiao, Sherry Yang, Wenhao Yu, and Allan Zhou. Evaluating gemini robot...

    2025

  61. [61]

    Causal video models are data-efficient robot policy learners.Rhoda AI Blog, 2026

    Rhoda AI Team. Causal video models are data-efficient robot policy learners.Rhoda AI Blog, 2026

    2026

  62. [62]

    Understanding physical dynamics with counterfactual world modeling

    Rahul Venkatesh, Honglin Chen, Kevin Feigelis, Daniel M Bear, Khaled Jedoui, Klemen Kotar, Felix Binder, Wanhee Lee, Sherry Liu, Kevin A Smith, et al. Understanding physical dynamics with counterfactual world modeling. InEuropean Conference on Computer Vision, pages 368–387. Springer, 2024

    2024

  63. [63]

    Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314, 2025

    Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, et al. Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314, 2025

    Pith/arXiv arXiv 2025

  64. [64]

    Eva: Aligning video world models with executable robot actions via inverse dynamics rewards, 2026

    Ruixiang Wang, Qingming Liu, Yueci Deng, Guiliang Liu, Zhen Liu, and Kui Jia. Eva: Aligning video world models with executable robot actions via inverse dynamics rewards, 2026. 14

    2026

  65. [65]

    Interactive world simulator for robot policy training and evaluation.arXiv preprint arXiv:2603.08546, 2026

    Yixuan Wang, Rhythm Syed, Fangyu Wu, Mengchao Zhang, Aykut Onol, Jose Barreiros, Hooshang Nayyeri, Tony Dear, Huan Zhang, and Yunzhu Li. Interactive world simulator for robot policy training and evaluation.arXiv preprint arXiv:2603.08546, 2026

    arXiv 2026

  66. [66]

    Precise action-to-video generation through visual action prompts

    Yuang Wang, Chao Wen, Haoyu Guo, Sida Peng, Minghan Qin, Hujun Bao, Xiaowei Zhou, and Ruizhen Hu. Precise action-to-video generation through visual action prompts. InICCV, 10 2025

    2025

  67. [67]

    Wolpert and J

    Daniel M. Wolpert and J. Randall Flanagan. Motor prediction.Current Biology, 11(18): R729–R732, 2001. doi: 10.1016/S0960-9822(01)00432-8

    work page doi:10.1016/s0960-9822(01)00432-8 2001

  68. [68]

    Wolpert, Zoubin Ghahramani, and Michael I

    Daniel M. Wolpert, Zoubin Ghahramani, and Michael I. Jordan. An internal model for sensori- motor integration.Science, 269(5232):1880–1882, 1995. doi: 10.1126/science.7569931

    doi:10.1126/science.7569931 1995

  69. [69]

    Wolpert, R

    Daniel M. Wolpert, R. Chris Miall, and Mitsuo Kawato. Internal models in the cerebellum. Trends in Cognitive Sciences, 2(9):338–347, 1998. doi: 10.1016/S1364-6613(98)01221-2

    doi:10.1016/s1364-6613(98)01221-2 1998

  70. [70]

    Sun, Ashley Neall, Tong Wu, Shengqu Cai, and Gordon Wetzstein

    Linxi Xie, Lisong C. Sun, Ashley Neall, Tong Wu, Shengqu Cai, and Gordon Wetzstein. Generated reality: Human-centric world simulation using interactive video generation with hand and camera control.arXiv preprint arXiv:2602.18422, 2026

    arXiv 2026

  71. [71]

    Kinema4d: Kinematic4d world modeling for spatiotemporal embodied simulation.arXiv preprint arXiv:2603.16669, 2026

    Mutian Xu, Tianbao Zhang, Tianqi Liu, Zhaoxi Chen, Xiaoguang Han, and Ziwei Liu. Kinema4d: Kinematic4d world modeling for spatiotemporal embodied simulation.arXiv preprint arXiv:2603.16669, 2026

    arXiv 2026

  72. [72]

    RoboPanoptes: The All-Seeing Robot with Whole-body Dexterity

    Xiaomeng Xu, Dominik Bauer, and Shuran Song. RoboPanoptes: The All-Seeing Robot with Whole-body Dexterity. InProceedings of Robotics: Science and Systems, LosAngeles, CA, USA, June 2025. doi: 10.15607/RSS.2025.XXI.042

    doi:10.15607/rss.2025.xxi.042 2025

  73. [73]

    Learning interactive real-world simulators.arXiv preprint arXiv:2310.06114, 2023

    Sherry Yang, Yilun Du, Kamyar Ghasemipour, Jonathan Tompson, Leslie Kaelbling, Dale Schuurmans, and Pieter Abbeel. Learning interactive real-world simulators.arXiv preprint arXiv:2310.06114, 2023

    Pith/arXiv arXiv 2023

  74. [74]

    Orv: 4d occupancy-centric robot video generation, 2025

    Xiuyu Yang, Bohan Li, Shaocong Xu, Nan Wang, Chongjie Ye, Zhaoxi Chen, Minghan Qin, Yikang Ding, Zheng Zhu, Xin Jin, Hang Zhao, and Hao Zhao. Orv: 4d occupancy-centric robot video generation, 2025

    2025

  75. [75]

    Real-to-sim robot policy evaluation with gaussian splatting simulation of soft-body interactions

    Kaifeng Zhang, Shuo Sha, Hanxiao Jiang, Matthew Loper, Hyunjong Song, Guangyan Cai, Zhuo Xu, Xiaochen Hu, Changxi Zheng, and Yunzhu Li. Real-to-sim robot policy evaluation with gaussian splatting simulation of soft-body interactions. InProceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2026

    2026

  76. [76]

    Veo-act: How far can frontier video models advance generalizable robot manipulation?, 2026

    Zhongru Zhang, Chenghan Yang, Qingzhou Lu, Yanjiang Guo, Jianke Zhang, Yucheng Hu, and Jianyu Chen. Veo-act: How far can frontier video models advance generalizable robot manipulation?, 2026

    2026

  77. [77]

    Learning fine-grained bimanual manipulation with low-cost hardware.arXiv preprint arXiv:2304.13705, 2023

    Tony Z Zhao, Vikash Kumar, Sergey Levine, and Chelsea Finn. Learning fine-grained bimanual manipulation with low-cost hardware.arXiv preprint arXiv:2304.13705, 2023

    Pith/arXiv arXiv 2023

  78. [78]

    Tesseract: learning 4d embodied world models.arXiv preprint arXiv:2504.20995, 2025

    Haoyu Zhen, Qiao Sun, Hongxin Zhang, Junyan Li, Siyuan Zhou, Yilun Du, and Chuang Gan. Tesseract: learning 4d embodied world models.arXiv preprint arXiv:2504.20995, 2025

    Pith/arXiv arXiv 2025

  79. [79]

    Action images: End-to-end policy learning via multiview video generation, 2026

    Haoyu Zhen, Zixian Gao, Qiao Sun, Yilin Zhao, Yuncong Yang, Yilun Du, Pengsheng Guo, Tsun-Hsuan Wang, Yi-Ling Qiao, and Chuang Gan. Action images: End-to-end policy learning via multiview video generation, 2026

    2026

  80. [80]

    3dflowaction: Learning cross-embodiment manipulation from 3d flow world model, 2025

    Hongyan Zhi, Peihao Chen, Siyuan Zhou, Yubo Dong, Quanxi Wu, Lei Han, and Mingkui Tan. 3dflowaction: Learning cross-embodiment manipulation from 3d flow world model, 2025

    2025

Showing first 80 references.