Pith. sign in

REVIEW 5 major objections 5 minor 3 cited by

Robot video-action models should be pretrained natively from scratch—semantic tokenizer and causal backbone together—on web-scale video, yielding few-shot generalization and 225 Hz closed-loop control.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 07:47 UTC pith:RNHARHC7

load-bearing objection A serious systems paper whose headline claim about web-video action supervision is unsupported by its own data recipe and experiments. the 5 major comments →

arxiv 2607.08639 v2 pith:RNHARHC7 submitted 2026-07-09 cs.RO cs.CV

Native Video-Action Pretraining for Generalizable Robot Control

Qihang Zhang , Lin Li , Luyao Zhang , Shuai Yang , Yiming Luo , Shuaiting Li , Ruilin Wang , Junke Wang
show 21 more authors
Jiahao Shao Gangwei Xu Jiaming Zhou Yishu Shen Yudong Jin Fangyi Xu Shuailei Ma Jiaqi Liao Guanxing Lu Zifan Shi Yongkun Wen Yujie Zhao Weixuan Tang Xinyang Wang Chaojian Li Jiapeng Zhu Ka Leong Cheng Nan Xue Xing Zhu Yujun Shen Yinghao Xu
This is my paper
classification cs.RO cs.CV
keywords video-action pretrainingrobot foundation modelslatent actionssemantic visual-action tokenizercausal diffusion transformersparse mixture-of-expertsfew-shot generalizationreal-time closed-loop control
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

LingBot-VA 2.0 argues that repurposing video generative models designed for digital content creation is inherently inadequate for physical environments. The paper's central claim is that the full video-action stack—a semantic visual-action tokenizer plus a causal diffusion transformer—should be pretrained from scratch for robot control on web-scale image and video data, so that world states and actions share one latent space and temporal dynamics are learned in causal form. If true, generalist robot policies can be built from abundant unlabeled video plus modest robot data rather than by retrofitting generic video generators, with better generalization and real-time closed-loop control. The paper supports this with real-world manipulation results and simulation, reporting few-shot adaptation from 10–15 demonstrations, zero-shot execution in some settings, and a peak asynchronous control frequency of 225 Hz.

Core claim

LingBot-VA 2.0 is a video-action foundation model built natively for embodiment. Its semantic visual-action tokenizer aligns reconstruction latents with a frozen visual foundation model and learns latent actions from unlabeled video through inverse and forward dynamics, putting world states and actions in one semantic latent space. On this space, a causal diffusion transformer with a sparse mixture-of-experts video stream is pretrained from scratch under a multi-task recipe that keeps text-to-image, text-to-video, video-action, in-context learning, and human–robot co-training objectives active throughout, with multi-chunk prediction to force trajectory-level dynamics. At deployment, Foresigh

What carries the argument

The central object is the semantic visual-action tokenizer: a video autoencoder whose latent space is pulled toward the features of a frozen perception encoder and which, in the same stage, learns compact latent actions by training an inverse-dynamics model (predicting a low-dimensional transition variable between consecutive latents) jointly with a forward-dynamics model (decoding that variable into a transport map plus residual). By forcing the transition variable through a bottleneck, the tokenizer makes unlabeled web video carry action-relevant supervision, and it is on this shared latent space that the causal DiT is pretrained from scratch.

Load-bearing premise

The load-bearing premise is that the compact transition variables learned from unlabeled video by watching frames change carry the information needed to control a real robot—the paper itself concedes the latent-action space is learned from passive video and could be sharpened by interaction—and no experiment isolates this assumption from the downstream robot-data post-training.

What would settle it

Freeze the trained tokenizer and probe whether a linear decoder from its latent transitions on unlabeled human video predicts the retargeted robot action chunks for the same manipulation; chance-level decoding would show the latent actions do not carry control-relevant information, undercutting the claim that web-scale video provides action supervision.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Video-action model pretraining no longer needs scarce robot data to acquire broad control priors: the action signal scales with web-scale unlabeled video.
  • Training the backbone causally from scratch avoids the bidirectional-to-causal retrofit that the paper argues erodes pretrained priors, aligning the model's temporal structure with closed-loop control.
  • Because world states and actions live in one semantic space, human demonstration videos can serve as in-context task prompts, enabling composition of unseen task arrangements without weight updates.
  • Human–robot co-training with retargeted hand poses adds a scalable, low-cost source of embodied data to the shared world model.
  • Asynchronous Foresight Reasoning with observation re-grounding hides most model latency behind robot motion, so the measured 225 Hz control frequency becomes achievable on real hardware.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If latent actions learned from passive video do transfer to robot action spaces, the same native-pretraining recipe could extend to other embodiments (humanoids, quadrupeds) and other sensor modalities, because the bottleneck forces the latent action to encode transition structure rather than appearance.
  • The paper does not isolate the web-video pretraining signal from downstream robot-data post-training; a controlled experiment with fixed downstream data that varies only the pretraining corpus would test whether the reported generalization margins come from native pretraining or from supervised fine-tuning.
  • Foresight Reasoning's predict-then-correct pattern suggests a general principle for asynchronous world-model control: any predictive rollout can be kept closed-loop by overwriting imagined latents with real observations as they arrive, a scheme that could generalize beyond this architecture.
  • The large multi-chunk prediction gains at 50 fps hint that myopic next-chunk supervision is a key bottleneck for learning dynamics from densely sampled video, a lesson that could apply to video-prediction models outside robotics.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper presents LingBot-VA 2.0, a video-action foundation model for robot manipulation built natively for control rather than adapted from generic video generators. Four design pillars are proposed: a semantic visual-action tokenizer that aligns visual latents with a frozen foundation model and learns latent actions from unlabeled video; a causal Diffusion Transformer trained from scratch with a sparse MoE video stream; multi-chunk prediction (MCP) for trajectory-level supervision; and Foresight Reasoning, an asynchronous inference scheme that hides model latency behind execution while re-grounding on real observations. The system is evaluated on RoboTwin 2.0, on a small real-world benchmark against π0.5 and LingBot-VA, and with ablations for the tokenizer and MCP. The paper claims few-shot/zero-shot adaptation and a peak asynchronous control frequency of 225 Hz.

Significance. If the central claim were fully supported, the paper would make an important contribution: it would show that a video-action stack pretrained natively with causal structure and semantic latents can outperform retrofitted video generators and VLA baselines, and that web video can supply action-relevant supervision. The paper is also valuable for its concrete engineering results: the tokenizer ablation (Table 2) and MCP ablation (Figure 10) are useful controlled comparisons, and the inference-acceleration table (Table 3) documents a systematic path to real-time control. However, as it stands the evidence does not isolate the headline mechanism — web-scale action supervision from unlabeled video — from other components such as the tokenizer, MoE, human co-training, or the added robot data. The significance is therefore conditional on additional experiments or a substantial revision of the claims.

major comments (5)
  1. [Sec. 3.1–3.2, Eq. (5)] The paper's central claim — that unlabeled web video supplies action-relevant supervision and that control knowledge scales to web-video size — is not implemented in the described recipe. In Sec. 3.1, web-scale corpora are used only for T2I/T2V; the TI2VA objective of Eq. (5) is trained on robot data (Sec. 3.2), human data (Sec. 3.3), and ICL pairs (Sec. 3.4). No experiment removes or varies the web pretraining signal, so the reported gains over π0.5 and LingBot-VA are confounded with the tokenizer, MoE, human co-training, and thousands of hours of added robot demonstrations. Please either (a) add an ablation that trains with/without web T2I/T2V, or with a video-action objective applied to web video, or (b) revise the claims to state that web data are used only for image/video priors, not for action supervision.
  2. [Sec. 2.2.2, Sec. 2.3.2, Sec. 4.1.1] The mapping from latent actions ℓ_t to deployed motor commands is unspecified. Sec. 2.2.2 defines ℓ_t as a compact transition variable with d_ℓ ≪ dim(z_t), then writes a_t ≡ ℓ_t to match Eq. (5). However, Sec. 4.1.1 describes the action output head as a linear layer 768→30, i.e., producing raw 30-dimensional action chunks. If the pretraining targets are latent actions, how are they decoded to the 30-dim motor commands executed by the robot? If the targets are raw actions, what role do the latent actions play in the video-action objective? Without this mapping, the few-shot and zero-shot transfer claims are not fully defined.
  3. [Table 1, Fig. 8] The main results lack uncertainty information. Table 1 reports point success rates without seeds, variance, or trial counts; Fig. 8 shows success/progress bars with no sample sizes or confidence intervals. Given the small reported gaps (e.g., 93.8 vs 92.9 in Table 1 clean; 82.7 vs 93.8 for π0.5 in Fig. 8), the reader cannot judge whether the improvements are significant. Please report the number of rollouts, seeds, and error bars for both simulation and real-world results.
  4. [Table 2, Sec. 4.4] The tokenizer ablation does not isolate the web-video action-supervision mechanism. It compares a reconstruction VAE and the proposed tokenizer on a 1.3B model with identical web T2I/T2V pretraining and identical robot post-training; both variants use the same video-action objective on robot data. This shows that the semantic tokenizer helps, but it does not show that latent actions learned from unlabeled web video transfer to control. In addition, there is no direct comparison against a DVA-style native-causal baseline, so the 'causal from scratch' claim is not separated from the MoE, MCP, or human co-training components.
  5. [Sec. 2.3.7, Eq. (30)] Foresight Reasoning is introduced with a new forward-dynamics grounding loss (Eq. 30), but no ablation measures its contribution to closed-loop success. The reported 225 Hz figure uses K=32 in the formula (1000/t_chunk)×K, yet the relationship between K, the latent-frame chunk size (1–4), and the 30-dimensional action chunk is not made precise. Please report rollouts with and without Foresight Reasoning and clarify the relationship between chunk definitions and the asynchronous frequency.
minor comments (5)
  1. [Sec. 1] Typo: 'anative route' should be 'a native route'.
  2. [Throughout] The spacing in 'LingBot-V A' is inconsistent (sometimes 'LingBot-VA'); please unify the notation.
  3. [Figure 10] The MCP ablation would be clearer with final-value error bars and a statement of how many seeds were used; the '29.7 percentage points' advantage at 5k steps appears to shrink at convergence.
  4. [Sec. 2.2.2 and Sec. 2.3.7] The same symbol f_ψ is used for the frozen tokenizer FDM and later for the policy's online FDM_θ; please distinguish these to avoid confusion.
  5. [Eq. (21)] The approximation of the joint multi-chunk distribution by a product of conditionals is plausible but not discussed; a sentence on why the factorization is valid here would help.

Circularity Check

0 steps flagged

No significant circularity: the system is empirically evaluated against external benchmarks, though the headline web-video action-supervision claim is under-isolated rather than derived.

full rationale

LingBot-VA 2.0 is an empirical systems paper, not a formal derivation chain: its equations (e.g., Eq. 5–7, 9–14, 22–30) define training objectives, and no reported result is obtained by algebraically transforming its own assumptions. The main evaluations compare against external baselines (π0.5, X-VLA, Motus, WAN2.2 VAE, RoboTwin) on held-out metrics, and the tokenizer ablation (Tab. 2) and MCP ablation (Fig. 10) vary exactly one component while holding downstream training fixed. Citations to the authors' prior work (RepWAM [99], Next Forcing [112], LingBot-VA [50], LingBot-Video [70], Zero-WAM [60]) are method adoptions rather than load-bearing external proofs; none is invoked as a uniqueness theorem that forces the paper's conclusions. The closest issue is a support gap, not a loop: Sec. 3.1 assigns web-scale data only to T2I/T2V, while TI2VA/ICL/HCT are trained on robot and human data, so the claim in Sec. 1 that 'its action signal no longer depends on scarce robot demonstrations' is not experimentally isolated. The tokenizer's 'latent action' is defined as an inverse-dynamics transition variable (Sec. 2.2.2, Eq. 12–14) and identified with a_t only for notation; the necessary transfer to raw 30-dim robot actions is not fully specified, and the paper itself concedes in Sec. 6 that the 'latent-action space is learned from passive video, and interactive or reinforcement signals could sharpen it toward control.' That is a missing experiment / underspecified transfer, not an equation-level self-reference. Score 2 reflects the under-tested central claim and heavy author-group self-citation, without any step that reduces to its inputs by construction.

Axiom & Free-Parameter Ledger

6 free parameters · 6 axioms · 0 invented entities

The paper introduces no new physical entities or forces. The 'latent action' space and imagined FDM latents are learned internal representations, not new postulated entities with independent falsifiable handles. The chief burden is therefore not invented entities but the domain assumptions listed above, especially that unlabeled-video latent actions transfer to real robot control.

free parameters (6)
  • λ_align = not reported
    Weight of the semantic alignment loss in Eq. (11); chosen by validation but no value or tuning procedure is given.
  • λ_act = not reported
    Weight combining action loss into L_VA in Sec. 2.1.2; no value or ablation.
  • MCP horizon weights (w1,w2,w3) = (0.5, 0.2, 0.1) from Next Forcing
    Hand-set in Eq. (23); inherited from prior work without a dedicated ablation.
  • Timestep shifts per modality = image 2, web/video 2, robot 5, MCP 10, action 1
    Chosen by design in Sec. 4.1.2; affects the flow-matching training dynamics of every stage.
  • Chunk size and attention window at inference = chunk size 2, window 64
    Selected from a training-time resampling range of 1-4 chunks and 1-64 windows (Sec. 4.1.1), not justified by experiments.
  • K = low-level steps per chunk = 32
    Used in Table 3 to convert 142 ms/chunk into 225 Hz asynchronous frequency; if K is smaller, the claimed frequency drops proportionally.
axioms (6)
  • domain assumption Latent actions ℓ_t extracted by inverse dynamics from unlabeled video capture control-relevant transitions and can serve as action supervision for downstream robot control.
    Central to native pretraining; stated in Sec. 1 and Sec. 2.2.2 (Eq. 12-14) but never isolated from robot-data post-training in the experiments.
  • domain assumption Clip-level features of the frozen Perception Encoder are a suitable semantic target for control-oriented visual representations.
    Used in Eq. (10) as the semantic alignment objective; no ablation varies the teacher model.
  • domain assumption Causal next-latent flow-matching pretraining on web video, with multi-chunk prediction, transfers to embodied action prediction.
    The paper's core design premise (Sec. 2.3); the transfer is asserted but not directly tested against a bidirectional-retrofit baseline under the same tokenizer.
  • domain assumption Human hand poses retargeted through Φ to gripper apertures preserve enough dexterity to supervise robot manipulation.
    Used in Sec. 2.3.5 (Eq. 25) for human-robot co-training; no quantitative evaluation of retargeting quality.
  • domain assumption The policy's own video expert can act as a forward-dynamics model for Foresight Reasoning, and re-grounding corrects drift before it accumulates.
    Assumed in Sec. 2.3.7 (Eq. 29-30); the FDM loss is trained but the fidelity of imagined latents is not measured.
  • standard math Flow matching / rectified-flow and consistency distillation objectives are valid training procedures.
    Standard generative-modeling results from Lipman et al. and Song et al.; accepted as background.

pith-pipeline@v1.3.0-alltime-deepseek · 29769 in / 13521 out tokens · 136270 ms · 2026-08-02T07:47:06.005183+00:00 · methodology

0 comments
read the original abstract

The advent of video-action models offers a promising path for robot control. Nevertheless, we argue that repurposing video generative models designed for digital content creation is inherently inadequate for physical environments. To bridge this gap, we present LingBot-VA 2.0, a video-action foundation model built from the ground up for embodiment. Four core design principles showcase its evolution from LingBot-VA. (1) Departing from traditional reconstruction-focused VAEs, we introduce a semantic visual-action tokenizer, which aligns visual representations with both semantics and actions, improving instruction following and action precision in subsequent policy learning. (2) Given the strictly causal nature of temporal dynamics, we adopt a causal pretraining paradigm, training from scratch to circumvent the catastrophic forgetting that frequently occurs when adapting bidirectional architectures. (3) To meet the demands of high-frequency inference, our model employs a sparse MoE backbone, expanding model capacity without compromising efficiency. (4) Real-time closed-loop control is realized through an enhanced asynchronous inference scheme, which predicts future latents in parallel with action execution while re-grounding each rollout on the latest observation via learned forward dynamics. Real-world deployment validates LingBot-VA 2.0 as a robust foundation model, as evidenced by its few-shot generalization across complex manipulation tasks.

Figures

Figures reproduced from arXiv: 2607.08639 by Chaojian Li, Fangyi Xu, Gangwei Xu, Guanxing Lu, Jiahao Shao, Jiaming Zhou, Jiapeng Zhu, Jiaqi Liao, Junke Wang, Ka Leong Cheng, Lin Li, Luyao Zhang, Nan Xue, Qihang Zhang, Ruilin Wang, Shuailei Ma, Shuaiting Li, Shuai Yang, Weixuan Tang, Xing Zhu, Xinyang Wang, Yiming Luo, Yinghao Xu, Yishu Shen, Yongkun Wen, Yudong Jin, Yujie Zhao, Yujun Shen, Zifan Shi.

Figure 1
Figure 1. Figure 1: Overview of LingBot-VA 2.0. The Planner (Sec. 2.3.1) decomposes long-horizon goals into structured subtask context. The semantic visual-action tokenizer (Sec. 2.2) encodes image observations and human video prompts (Sec. 2.3.4) as visual latents. The video model with the MoE (Sec. 2.3.2) and the action model autoregressively predict future visual latents and robot actions. MCP (Sec. 2.3.3) adds future-chun… view at source ↗
Figure 2
Figure 2. Figure 2: Semantic visual-action tokenizer. The visual tokenizer aligns reconstruction-oriented latents with visual foundation￾model features, while the latent action tokenizer learns compact transition variables through inverse and forward dynamics. Adapted from RepWAM [98]. 2.2.1 Visual Tokenization with Semantic Alignment We adopt a Vision Transformer (ViT) autoencoder for visual tokenization [25, 97]. Given a vi… view at source ↗
Figure 3
Figure 3. Figure 3: Dual-system hierarchical policy. A high-level VLM planner consumes the task goal and sparse recent observations to produce structured subtask context, which conditions the low-level video-action policy together with robot state to produce closed-loop actions. where (Kt, δt) are predicted from ℓt. The transport Kt moves information across spatial latent tokens, while δt accounts for changes that cannot be e… view at source ↗
Figure 4
Figure 4. Figure 4: Training loss comparison between the MoE-13B-A1.9B video-stream model and the Dense-5B baseline. Left: loss aligned by optimization steps, where both models remain close and the dense baseline is slightly lower. Right: loss aligned by wall-clock training time, where the final losses nearly coincide. where σ(·) is the sigmoid function and Ne is the number of routed experts. We use group-limited top-k routin… view at source ↗
Figure 5
Figure 5. Figure 5: Multi-Chunk Prediction (MCP). Lightweight auxiliary modules predict several future latent chunks at increasing horizons from the main model’s representation, providing dense temporal supervision during training. Adapted from Next Forcing [111]. chunk, creating a myopic supervision problem in which the learning signal becomes overly local: adjacent chunks, especially at high frame rates, are often visually … view at source ↗
Figure 6
Figure 6. Figure 6: Foresight Reasoning as an asynchronous inference/execution flow. A cold start initializes the KV cache and pre￾computes the first action chunk. During the loop, the executor runs the current chunk while real observations enter OBSQUEUE; the inference branch re-grounds the KV cache with those observations, appends the currently executing action, imagines its visual outcome with FDM grounding, and predicts t… view at source ↗
Figure 7
Figure 7. Figure 7: In-context human-robot data curation pipeline. Robot videos are sampled by task semantics, converted into human demonstration prompts with a VLM, synthesized as human videos, and filtered before pairing with the original robot trajectories as ICL samples. dataset encompasses data from over 10 robot video pre-training datasets (e.g., AgiBot [2], OXE [73]), comprising more than 5,000 tasks and over 50,000 sa… view at source ↗
Figure 8
Figure 8. Figure 8: Real-world deployment results. Success rate and task progress rate across real-world tasks, comparing LingBot-VA 2.0 against representative baselines. task-level context across multiple interaction steps, while its video grounding also improves the precision of local manipulation. Compared with LingBot-VA, the gains come from causal video-action pretraining, which exposes the model to more generalizable vi… view at source ↗
Figure 9
Figure 9. Figure 9: In-context learning demo. We demonstrate ICL rollouts on four unseen tasks. For each rollout, given a reference video demonstration as the task instruction and the robot observation, LingBot-VA 2.0 transfers the same action procedure to robot manipulation without parameter updates [PITH_FULL_IMAGE:figures/full_fig_p019_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Multi-chunk prediction ablation. Task success rate (%) on RoboTwin across training steps. MCP converges faster and reaches higher final accuracy than the baseline at both 12 and 50 fps. and optimization setup, with MCP enabled in only one of the two variants. MCP consistently improves optimization efficiency, converging faster and reaching higher final task success at both 12 and 50 fps. The advantage is … view at source ↗
Figure 11
Figure 11. Figure 11: Real-world demonstrations. Qualitative rollouts of LingBot-VA 2.0 across four representative tasks, covering long-horizon tabletop organization, interaction with moving objects, fine-grained grasping, and reactive visual control. 5 Related Work 5.1 Generalist robot policies and VLA models Vision-language-action (VLA) models directly map visual observations and language instructions to actions, inheriting … view at source ↗

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Enfold: Folding World-Generator Computation into Predictive Representations for Efficient Embodied Control

    cs.RO 2026-07 conditional novelty 6.0

    Distilling a video generator's multi-level hidden states into a current-only representation lets a robot act without executing the generator, cutting action latency 3.7–10.1x at comparable success rates.

  2. WorldScape Policy 2.0: Empowering Steerable World Action Modeling with Reasoning-Augmented Memory

    cs.RO 2026-07 conditional novelty 6.0

    A controllable world-action model with reasoning-augmented long short-term memory and event-grounded pretraining improves long-horizon robot manipulation.

  3. Xiaomi-Robotics-1: Scaling Vision-Language-Action Models with over 100K Hours of Real-World Trajectories

    cs.RO 2026-07 conditional novelty 6.0

    Pre-training a VLA model on 100k hours of auto-labeled UMI trajectories, then post-training on robot data, yields SOTA simulated manipulation and data-efficient fine-tuning.

Reference graph

Works this paper leans on

134 extracted references · 53 linked inside Pith · cited by 3 Pith papers

  1. [1]

    1x world model: From video to action

    1X Technologies. 1x world model: From video to action. https://www.1x.tech/discover/world-model-self-learning,

  2. [2]

    Agibot world colosseo: A large-scale manipulation platform for scalable and intelligent embodied systems.arXiv preprint arXiv:2503.06669, 2025

    AgiBot-World-Contributors, Qingwen Bu, Jisong Cai, Li Chen, Xiuqi Cui, Yan Ding, Siyuan Feng, Shenyuan Gao, Xindong He, Xuan Hu, Xu Huang, Shu Jiang, Yuxin Jiang, Cheng Jing, Hongyang Li, et al. Agibot world colosseo: A large-scale manipulation platform for scalable and intelligent embodied systems.arXiv preprint arXiv:2503.06669, 2025

    Pith/arXiv arXiv 2025

  3. [3]

    Video pretraining (vpt): Learning to act by watching unlabeled online videos

    Bowen Baker, Ilge Akkaya, Peter Zhokov, Joost Huizinga, Jie Tang, Adrien Ecoffet, Brandon Houghton, Raul Sampedro, and Jeff Clune. Video pretraining (vpt): Learning to act by watching unlabeled online videos. InAdv. Neural Inform. Process. Syst., 2022

    2022

  4. [4]

    One transformer fits all distributions in multi-modal diffusion at scale

    Fan Bao, Shen Nie, Kaiwen Xue, Chongxuan Li, Shi Pu, Yaole Wang, Gang Yue, Yue Cao, Hang Su, and Jun Zhu. One transformer fits all distributions in multi-modal diffusion at scale. InInt. Conf. Mach. Learn., 2023

    2023

  5. [5]

    Navigation world models

    Amir Bar, Gaoyue Zhou, Danny Tran, Trevor Darrell, and Yann LeCun. Navigation world models. InIEEE Conf. Comput. Vis. Pattern Recog., 2025

    2025

  6. [6]

    A careful examination of large behavior models for multitask dexterous manipulation.arXiv preprint arXiv:2507.05331, 2025

    Jose Barreiros, Andrew Beaulieu, Aditya Bhat, Rick Cory, Eric Cousineau, Hongkai Dai, Ching-Hsin Fang, Kunimatsu Hashimoto, Muhammad Zubair Irshad, Masha Itkina, et al. A careful examination of large behavior models for multitask dexterous manipulation.arXiv preprint arXiv:2507.05331, 2025

    Pith/arXiv arXiv 2025

  7. [7]

    Gen2act: Human video generation in novel scenarios enables generalizable robot manipulation

    Homanga Bharadhwaj, Debidatta Dwibedi, Abhinav Gupta, Shubham Tulsiani, Carl Doersch, Ted Xiao, Dhruv Shah, Fei Xia, Dorsa Sadigh, and Sean Kirmani. Gen2act: Human video generation in novel scenarios enables generalizable robot manipulation. InConference on Robot Learning (CoRL), 2024. 23

    2024

  8. [8]

    Motus: A unified latent action world model.arXiv preprint arXiv:2512.13030, 2025

    Hongzhe Bi, Hengkai Tan, Shenghao Xie, Zeyuan Wang, Shuhe Huang, Haitian Liu, Ruowen Zhao, Yao Feng, Chendong Xiang, Yinze Rong, Hongyan Zhao, Hanyu Liu, Zhizhong Su, Lei Ma, Hang Su, et al. Motus: A unified latent action world model.arXiv preprint arXiv:2512.13030, 2025

    Pith/arXiv arXiv 2025

  9. [9]

    π0: A vision- language-action flow model for general robot control

    Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, Szymon Jakubczak, Tim Jones, Liyiming Ke, Sergey Levine, Adrian Li-Bell, et al. π0: A vision- language-action flow model for general robot control. InRobotics: Science and Systems, 2025

    2025

  10. [10]

    Perception encoder: The best visual embeddings are not at the output of the network

    Daniel Bolya, Po-Yao Huang, Peize Sun, Jang Hyun Cho, Andrea Madotto, Chen Wei, Tengyu Ma, Jiale Zhi, Jathushan Rajasegaran, Hanoona Bangalath, et al. Perception encoder: The best visual embeddings are not at the output of the network. InNeurIPS, 2025

    2025

  11. [11]

    Rt-2: Vision-language-action models transfer web knowledge to robotic control

    Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choromanski, Tianli Ding, Danny Driess, Avinava Dubey, Chelsea Finn, Pete Florence, Chuyuan Fu, Montse Gonzalez Arenas, Keerthana Gopalakrishnan, Kehang Han, et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control. InConference on Robot Learning ...

    2023

  12. [12]

    Rt-1: Robotics transformer for real-world control at scale

    Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Jasmine Hsu, Julian Ibarz, Brian Ichter, Alex Irpan, Tomas Jackson, Sally Jesmonth, et al. Rt-1: Robotics transformer for real-world control at scale. InRobotics: Science and Systems, 2023

    2023

  13. [13]

    Genie: Generative interactive environments

    Jake Bruce, Michael Dennis, Ashley Edwards, Jack Parker-Holder, Yuge Shi, Edward Hughes, Matthew Lai, Aditi Mavalankar, Richie Steigerwald, Chris Apps, Yusuf Aytar, Sarah Bechtle, Feryal Behbahani, Stephanie Chan, Nicolas Heess, et al. Genie: Generative interactive environments. InInt. Conf. Mach. Learn., 2024

    2024

  14. [14]

    Univla: Learning to act anywhere with task-centric latent actions

    Qingwen Bu, Yanting Yang, Jisong Cai, Shenyuan Gao, Guanghui Ren, Maoqing Yao, Ping Luo, and Hongyang Li. Univla: Learning to act anywhere with task-centric latent actions. InRobotics: Science and Systems, 2025

    2025

  15. [15]

    Worldvla: Towards autoregressive action world model.arXiv preprint arXiv:2506.21539, 2025

    Jun Cen, Chaohui Yu, Hangjie Yuan, Yuming Jiang, Siteng Huang, Jiayan Guo, Xin Li, Yibing Song, Hao Luo, Fan Wang, Deli Zhao, and Hao Chen. Worldvla: Towards autoregressive action world model.arXiv preprint arXiv:2506.21539, 2025

    Pith/arXiv arXiv 2025

  16. [16]

    Gamegen-x: Interactive open-world game video generation

    Haoxuan Che, Xuanhua He, Quande Liu, Cheng Jin, and Hao Chen. Gamegen-x: Interactive open-world game video generation. InInt. Conf. Learn. Represent., 2025

    2025

  17. [17]

    Gr-2: A generative video-language-action model with web-scale knowledge for robot manipulation.arXiv preprint arXiv:2410.06158, 2024

    Chi-Lam Cheang, Guangzeng Chen, Ya Jing, Tao Kong, Hang Li, Yifeng Li, Yuxiao Liu, Hongtao Wu, Jiafeng Xu, Yichu Yang, Hanbo Zhang, and Minzhao Zhu. Gr-2: A generative video-language-action model with web-scale knowledge for robot manipulation.arXiv preprint arXiv:2410.06158, 2024

    Pith/arXiv arXiv 2024

  18. [18]

    Diffusion forcing: Next-token prediction meets full-sequence diffusion

    Boyuan Chen, Diego Marti Monso, Yilun Du, Max Simchowitz, Russ Tedrake, and Vincent Sitzmann. Diffusion forcing: Next-token prediction meets full-sequence diffusion. InAdv. Neural Inform. Process. Syst., 2024

    2024

  19. [19]

    Robotwin 2.0: A scalable data generator and benchmark with strong domain randomization for robust bimanual robotic manipulation.arXiv preprint arXiv:2506.18088, 2025

    Tianxing Chen, Zanxin Chen, Baijun Chen, Zijian Cai, Yibin Liu, Zixuan Li, Qiwei Liang, Xianliang Lin, Yiheng Ge, Zhenyu Gu, Weiliang Deng, Yubin Guo, Tian Nian, Xuanbing Xie, Qiangyu Chen, et al. Robotwin 2.0: A scalable data generator and benchmark with strong domain randomization for robust bimanual robotic manipulation.arXiv preprint arXiv:2506.18088, 2025

    Pith/arXiv arXiv 2025

  20. [20]

    Moto: Latent motion token as the bridging language for learning robot manipulation from videos

    Yi Chen, Yuying Ge, Weiliang Tang, Yizhuo Li, Yixiao Ge, Mingyu Ding, Ying Shan, and Xihui Liu. Moto: Latent motion token as the bridging language for learning robot manipulation from videos. InInt. Conf. Comput. Vis., 2025

    2025

  21. [21]

    Diffusion policy: Visuomotor policy learning via action diffusion

    Cheng Chi, Siyuan Feng, Yilun Du, Zhenjia Xu, Eric Cousineau, Benjamin Burchfiel, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. InRobotics: Science and Systems, 2023

    2023

  22. [22]

    Universal manipulation interface: In-the-wild robot teaching without in-the-wild robots

    Cheng Chi, Zhenjia Xu, Chuer Pan, Eric Cousineau, Benjamin Burchfiel, Siyuan Feng, Russ Tedrake, and Shuran Song. Universal manipulation interface: In-the-wild robot teaching without in-the-wild robots. InRobotics: Science and Systems, 2024

    2024

  23. [23]

    Damai Dai, Chengqi Deng, Chenggang Zhao, R. X. Xu, Huazuo Gao, Deli Chen, Jiashi Li, Wangding Zeng, Xingkai Yu, Y . Wu, Zhenda Xie, Y . K. Li, Panpan Huang, Fuli Luo, Chong Ruan, Zhifang Sui, and Wenfeng Liang. DeepSeekMoE: Towards ultimate expert specialization in mixture-of-experts language models. InProceedings of the Annual Meeting of the Association ...

    2024

  24. [24]

    DeepSeek-V3 technical report.arXiv preprint arXiv:2412.19437, 2024

    DeepSeek-AI. DeepSeek-V3 technical report.arXiv preprint arXiv:2412.19437, 2024

    Pith/arXiv arXiv 2024

  25. [25]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. InInt. Conf. Learn. Represent., 2021. 24

    2021

  26. [26]

    Tenenbaum, Dale Schuurmans, and Pieter Abbeel

    Yilun Du, Mengjiao Yang, Bo Dai, Hanjun Dai, Ofir Nachum, Joshua B. Tenenbaum, Dale Schuurmans, and Pieter Abbeel. Learning universal policies via text-guided video generation. InAdv. Neural Inform. Process. Syst., 2023

    2023

  27. [27]

    Learning universal policies via text-guided video generation.Advances in neural information processing systems, 36:9156–9172, 2023

    Yilun Du, Sherry Yang, Bo Dai, Hanjun Dai, Ofir Nachum, Josh Tenenbaum, Dale Schuurmans, and Pieter Abbeel. Learning universal policies via text-guided video generation.Advances in neural information processing systems, 36:9156–9172, 2023

    2023

  28. [28]

    Adaworld: Learning adaptable world models with latent actions

    Shenyuan Gao, Siyuan Zhou, Yilun Du, Jun Zhang, and Chuang Gan. Adaworld: Learning adaptable world models with latent actions. InInt. Conf. Mach. Learn., 2025

    2025

  29. [29]

    Infinite worlds with versatile interactions

    Zelin Gao, Qiuyu Wang, Jiapeng Zhu, Jingye Chen, Zichen Liu, Qingyan Bai, et al. Infinite worlds with versatile interactions. arXiv preprint arXiv:2607.07534, 2026

    Pith/arXiv arXiv 2026

  30. [30]

    Gemini robotics: Bringing ai into the physical world.arXiv preprint arXiv:2503.20020, 2025

    Gemini Robotics Team. Gemini robotics: Bringing ai into the physical world.arXiv preprint arXiv:2503.20020, 2025

    Pith/arXiv arXiv 2025

  31. [31]

    Gen-0: Embodied foundation models that scale with physical interaction

    Generalist AI. Gen-0: Embodied foundation models that scale with physical interaction. https://generalistai.com/blog/ nov-04-2025-GEN-0, 2025. Built on Harmonic Reasoning

    2025

  32. [32]

    Veo: A text-to-video generation system.Google DeepMind Technical Report, 2025

    Google DeepMind. Veo: A text-to-video generation system.Google DeepMind Technical Report, 2025

    2025

  33. [33]

    Mastering diverse control tasks through world models

    Danijar Hafner, Jurgis Pasukonis, Jimmy Ba, and Timothy Lillicrap. Mastering diverse control tasks through world models. Nature, 2025

    2025

  34. [34]

    Td-mpc2: Scalable, robust world models for continuous control

    Nicklas Hansen, Hao Su, and Xiaolong Wang. Td-mpc2: Scalable, robust world models for continuous control. InInt. Conf. Learn. Represent., 2024

    2024

  35. [35]

    Video prediction policy: A generalist robot policy with predictive visual representations

    Yucheng Hu, Yanjiang Guo, Pengchao Wang, Xiaoyu Chen, Yen-Jen Wang, Jianke Zhang, Koushil Sreenath, Chaochao Lu, and Jianyu Chen. Video prediction policy: A generalist robot policy with predictive visual representations. InInt. Conf. Mach. Learn., 2025

    2025

  36. [36]

    Enerverse: Envisioning embodied future space for robotics manipulation.arXiv preprint arXiv:2501.01895, 2025

    Siyuan Huang, Liliang Chen, Pengfei Liu, Yue Hu, Shengyu Zhang, Peng Gao, Hongsheng Li, Maoqing Yao, and Guanghui Ren. Enerverse: Envisioning embodied future space for robotics manipulation.arXiv preprint arXiv:2501.01895, 2025

    arXiv 2025

  37. [37]

    Self forcing: Bridging the train-test gap in autoregressive video diffusion.arXiv preprint arXiv:2506.08009, 2025

    Xun Huang, Zhengqi Li, Guande He, Mingyuan Zhou, and Eli Shechtman. Self forcing: Bridging the train-test gap in autoregressive video diffusion.arXiv preprint arXiv:2506.08009, 2025

    Pith/arXiv arXiv 2025

  38. [38]

    Nora: A small open-sourced generalist vision language action model for embodied tasks.arXiv preprint arXiv:2504.19854, 2025

    Chia-Yu Hung, Qi Sun, Pengfei Hong, Amir Zadeh, Chuan Li, U-Xuan Tan, Navonil Majumder, and Soujanya Poria. Nora: A small open-sourced generalist vision language action model for embodied tasks.arXiv preprint arXiv:2504.19854, 2025

    Pith/arXiv arXiv 2025

  39. [39]

    Dreamgen: Unlocking generalization in robot learning through video world models

    Joel Jang, Seonghyeon Ye, Zongyu Lin, et al. Dreamgen: Unlocking generalization in robot learning through video world models. InConference on Robot Learning (CoRL), 2025

    2025

  40. [40]

    Egomimic: Scaling imitation learning via egocentric video.arXiv preprint arXiv:2410.24221, 2024

    Simar Kareer, Dhruv Patel, Ryan Punamiya, Pranay Mathur, Shuo Cheng, Chen Wang, Judy Hoffman, and Danfei Xu. Egomimic: Scaling imitation learning via egocentric video.arXiv preprint arXiv:2410.24221, 2024

    Pith/arXiv arXiv 2024

  41. [41]

    Droid: A large-scale in-the-wild robot manipulation dataset

    Alexander Khazatsky, Karl Pertsch, Suraj Nair, Ashwin Balakrishna, Sudeep Dasari, Siddharth Karamcheti, Soroush Nasiriany, Mohan Kumar Srirama, Lawrence Yunliang Chen, Kirsty Ellis, Peter David Fagan, Joey Hejna, Masha Itkina, Marion Lepert, Yecheng Jason Ma, et al. Droid: A large-scale in-the-wild robot manipulation dataset. InRobotics: Science and Systems, 2024

    2024

  42. [42]

    Cosmos policy: Fine-tuning video models for visuomotor control and planning.arXiv preprint arXiv:2601.16163, 2026

    Moo Jin Kim, Yihuai Gao, Tsung-Yi Lin, Yen-Chen Lin, Yunhao Ge, Grace Lam, Percy Liang, Shuran Song, Ming-Yu Liu, Chelsea Finn, et al. Cosmos policy: Fine-tuning video models for visuomotor control and planning.arXiv preprint arXiv:2601.16163, 2026

    Pith/arXiv arXiv 2026

  43. [43]

    Openvla: An open-source vision-language-action model

    Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, Quan Vuong, Thomas Kollar, Benjamin Burchfiel, Russ Tedrake, Dorsa Sadigh, et al. Openvla: An open-source vision-language-action model. InConference on Robot Learning (CoRL), 2024

    2024

  44. [44]

    Auto-encoding variational bayes.arXiv preprint arXiv:1312.6114, 2013

    Diederik P Kingma and Max Welling. Auto-encoding variational bayes.arXiv preprint arXiv:1312.6114, 2013

    Pith/arXiv arXiv 2013

  45. [45]

    Kling-v3.https://kling.ai/, 2026

    KlingAI Team. Kling-v3.https://kling.ai/, 2026

    2026

  46. [46]

    Tenenbaum

    Po-Chen Ko, Jiayuan Mao, Yilun Du, Shao-Hua Sun, and Joshua B. Tenenbaum. Learning to act from actionless videos through dense correspondences. InInt. Conf. Learn. Represent., 2024

    2024

  47. [47]

    Deformnet: Latent space modeling and dynamics prediction for deformable object manipulation

    Chenchang Li, Zihao Ai, Tong Wu, Xiaosa Li, Wenbo Ding, and Huazhe Xu. Deformnet: Latent space modeling and dynamics prediction for deformable object manipulation. In2024 IEEE International Conference on Robotics and Automation (ICRA), pages 14770–14776. IEEE, 2024. 25

    2024

  48. [48]

    Cronusvla: Transferring latent motion across time for multi-frame prediction in manipulation.arXiv preprint arXiv:2506.19816, 2025

    Hao Li, Shuai Yang, Yilun Chen, Yang Tian, Xiaoda Yang, Xinyi Chen, Hanqing Wang, Tai Wang, Feng Zhao, Dahua Lin, et al. Cronusvla: Transferring latent motion across time for multi-frame prediction in manipulation.arXiv preprint arXiv:2506.19816, 2025

    arXiv 2025

  49. [49]

    Gr-3 technical report.arXiv preprint arXiv:2507.15493, 2025

    Jiacheng Li, Mengzhou Sun, Bowen Zhang, Zhe Zhao, Xiu Liu, et al. Gr-3 technical report.arXiv preprint arXiv:2507.15493, 2025

    Pith/arXiv arXiv 2025

  50. [50]

    Causal world modeling for robot control.arXiv preprint arXiv:2601.21998, 2026

    Lin Li, Qihang Zhang, Yiming Luo, Shuai Yang, Ruilin Wang, Fei Han, Mingrui Yu, Zelin Gao, Nan Xue, Xing Zhu, Yujun Shen, and Yinghao Xu. Causal world modeling for robot control.arXiv preprint arXiv:2601.21998, 2026

    Pith/arXiv arXiv 2026

  51. [51]

    Cogact: A foundational vision-language-action model for synergizing cognition and action in robotic manipulation.arXiv preprint arXiv:2411.19650, 2024

    Qixiu Li, Yaobo Liang, Zeyu Wang, Lin Luo, Xi Chen, Mozheng Liao, Fangyun Wei, Yu Deng, Sicheng Xu, Yizhong Zhang, Xiaofan Wang, Bei Liu, Jianlong Fu, Jianmin Bao, Dong Chen, Yuanchun Shi, Jiaolong Yang, and Baining Guo. Cogact: A foundational vision-language-action model for synergizing cognition and action in robotic manipulation.arXiv preprint arXiv:24...

    Pith/arXiv arXiv 2024

  52. [52]

    What matters when cotraining robot manipulation policies on everyday human videos?arXiv preprint arXiv:2606.06627, 2026

    Richard Li, Aditya Prakash, Andrew Wen, Saurabh Gupta, Yilun Du, and Pulkit Agrawal. What matters when cotraining robot manipulation policies on everyday human videos?arXiv preprint arXiv:2606.06627, 2026

    Pith/arXiv arXiv 2026

  53. [53]

    Wall-wm: Carving world action modeling at the event joints.arXiv preprint arXiv:2606.01955, 2026

    Shalfun Li, Victor Yao, Charles Yang, Truth Qu, Regis Cheng, Ryan Yu, Howard Lu, Newton V on, Vincent Chen, Yohann Tang, et al. Wall-wm: Carving world action modeling at the event joints.arXiv preprint arXiv:2606.01955, 2026

    Pith/arXiv arXiv 2026

  54. [54]

    Unified video action model

    Shuang Li, Yihuai Gao, Dorsa Sadigh, and Shuran Song. Unified video action model. InRobotics: Science and Systems, 2025

    2025

  55. [55]

    Vision-language foundation models as effective robot imitators

    Xinghang Li, Minghuan Liu, Hanbo Zhang, Cunjun Yu, Jie Xu, Hongtao Wu, Chilam Cheang, Ya Jing, Weinan Zhang, Huaping Liu, Hang Li, and Tao Kong. Vision-language foundation models as effective robot imitators. InInt. Conf. Learn. Represent., 2024

    2024

  56. [56]

    Propagation networks for model-based control under partial observation

    Yunzhu Li, Jiajun Wu, Jun-Yan Zhu, Joshua B Tenenbaum, Antonio Torralba, and Russ Tedrake. Propagation networks for model-based control under partial observation. In2019 International Conference on Robotics and Automation (ICRA), pages 1205–1211. IEEE, 2019

    2019

  57. [57]

    Dreamitate: Real-world visuomotor policy learning via video generation

    Junbang Liang, Ruoshi Liu, Ege Ozguroglu, Sruthi Sudhakar, Achal Dave, Pavel Tokmakov, Shuran Song, and Carl V ondrick. Dreamitate: Real-world visuomotor policy learning via video generation. InConference on Robot Learning (CoRL), 2024

    2024

  58. [58]

    Mixture-of-transformers: A sparse and scalable architecture for multi-modal foundation models.Transactions on Machine Learning Research, 2025

    Weixin Liang, LILI YU, Liang Luo, Srini Iyer, Ning Dong, Chunting Zhou, Gargi Ghosh, Mike Lewis, Wen tau Yih, Luke Zettlemoyer, and Xi Victoria Lin. Mixture-of-transformers: A sparse and scalable architecture for multi-modal foundation models.Transactions on Machine Learning Research, 2025

    2025

  59. [59]

    Genie envisioner: A unified world foundation platform for robotic manipulation.arXiv preprint arXiv:2508.05635, 2025

    Yue Liao, Pengfei Zhou, Siyuan Huang, Donglin Yang, Shengcong Chen, Yuxin Jiang, Yue Hu, Jingbin Cai, Si Liu, Jianlan Luo, Liliang Chen, Shuicheng Yan, Maoqing Yao, and Guanghui Ren. Genie envisioner: A unified world foundation platform for robotic manipulation.arXiv preprint arXiv:2508.05635, 2025

    Pith/arXiv arXiv 2025

  60. [60]

    Zero-wam: In-context world modeling for zero-shot task generalization

    LingBot-V A Team, RobbyAnt. Zero-wam: In-context world modeling for zero-shot task generalization. https://github. com/jiaming-zhou/Zero-W AM, 2026

    2026

  61. [61]

    Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. InInt. Conf. Learn. Represent., 2023

    2023

  62. [62]

    Efficient robotic policy learning via latent space backward planning

    Dongxiu Liu, Haoyi Niu, Zhihao Wang, Jinliang Zheng, Yinan Zheng, Zhonghong Ou, Jianming Hu, Jianxiong Li, and Xianyuan Zhan. Efficient robotic policy learning via latent space backward planning. InInt. Conf. Mach. Learn., 2025

    2025

  63. [63]

    Hybridvla: Collaborative diffusion and autoregression in a unified vision-language-action model.arXiv preprint arXiv:2503.10631, 2025

    Jiaming Liu, Hao Chen, Pengju An, Zhuoyang Liu, Renrui Zhang, Chenyang Gu, Xiaoqi Li, Ziyu Guo, Sixiang Chen, Mengzhen Liu, Chengkai Hou, Mengdi Zhao, KC alex Zhou, Pheng-Ann Heng, and Shanghang Zhang. Hybridvla: Collaborative diffusion and autoregression in a unified vision-language-action model.arXiv preprint arXiv:2503.10631, 2025

    Pith/arXiv arXiv 2025

  64. [64]

    Muon is scalable for LLM training.arXiv preprint arXiv:2502.16982, 2025

    Jingyuan Liu, Jianlin Su, Xingcheng Yao, Zhejun Jiang, Guokun Lai, Yulun Du, Yidao Qin, Weixin Xu, Enzhe Lu, Junjie Yan, Yanru Chen, Huabin Zheng, Yibo Liu, Shaowei Liu, Bohong Yin, Weiran He, Han Zhu, Yuzhi Wang, Jianzhou Wang, Mengnan Dong, Zheng Zhang, Yongsheng Kang, Hao Zhang, Xinran Xu, Yutao Zhang, Yuxin Wu, Xinyu Zhou, and Zhilin Yang. Muon is sca...

    Pith/arXiv arXiv 2025

  65. [65]

    Rdt-1b: a diffusion foundation model for bimanual manipulation

    Songming Liu, Lingxuan Wu, Bangguo Li, Hengkai Tan, Huayu Chen, Zhengyi Wang, Ke Xu, Hang Su, and Jun Zhu. Rdt-1b: a diffusion foundation model for bimanual manipulation. InInt. Conf. Learn. Represent., 2025

    2025

  66. [66]

    Being-h0: Vision-language-action pretraining from large-scale human videos.arXiv preprint arXiv:2507.15597, 2025

    Hao Luo, Yicheng Feng, Wanpeng Zhang, Sipeng Zheng, Ye Wang, Haoqi Yuan, Jiazheng Liu, Chaoyi Xu, Qin Jin, and Zongqing Lu. Being-h0: Vision-language-action pretraining from large-scale human videos.arXiv preprint arXiv:2507.15597, 2025. 26

    Pith/arXiv arXiv 2025

  67. [67]

    Being-h0.7: A latent world-action model from egocentric videos.arXiv preprint arXiv:2605.00078, 2026

    Hao Luo, Wanpeng Zhang, Yicheng Feng, Sipeng Zheng, Haiweng Xu, Chaoyi Xu, Ziheng Xi, Yuhui Fu, and Zongqing Lu. Being-h0.7: A latent world-action model from egocentric videos.arXiv preprint arXiv:2605.00078, 2026

    Pith/arXiv arXiv 2026

  68. [68]

    Deep learning for universal linear embeddings of nonlinear dynamics

    Bethany Lusch, J Nathan Kutz, and Steven L Brunton. Deep learning for universal linear embeddings of nonlinear dynamics. Nature communications, 9(1):4950, 2018

    2018

  69. [69]

    F1: A vision-language-action model bridging understanding and generation to actions.arXiv preprint arXiv:2509.06951, 2025

    Qi Lv, Weijie Kong, Hao Li, Jia Zeng, Zherui Qiu, Delin Qu, Haoming Song, Qizhi Chen, Xiang Deng, and Jiangmiao Pang. F1: A vision-language-action model bridging understanding and generation to actions.arXiv preprint arXiv:2509.06951, 2025

    Pith/arXiv arXiv 2025

  70. [70]

    Scaling mixture-of-experts video pretraining for embodied intelligence.arXiv preprint arXiv:2607.07675, 2026

    Shuailei Ma, Jiaqi Liao, Xinyang Wang, Jingjing Wang, Chaoran Feng, Zijing Hu, Chong Bao, Zichen Xi, Yuqi Gan, Weisen Wang, Yanhong Zeng, Qin Zhao, Zifan Shi, Wei Wu, Hao Ouyang, Qiuyu Wang, Shangzhan Zhang, Jiahao Shao, Yipengjing Sun, Liangxiao Hu, Lunke Pan, Nan Xue, Kecheng Zheng, Yinghao Xu, Xing Zhu, Yujun Shen, and Ka Leong Cheng. Scaling mixture-o...

    Pith/arXiv arXiv 2026

  71. [71]

    Gr00t n1: An open foundation model for generalist humanoid robots.arXiv preprint arXiv:2503.14734, 2025

    NVIDIA. Gr00t n1: An open foundation model for generalist humanoid robots.arXiv preprint arXiv:2503.14734, 2025

    Pith/arXiv arXiv 2025

  72. [72]

    Octo: An open-source generalist robot policy

    Octo Model Team, Dibya Ghosh, Homer Walke, Karl Pertsch, et al. Octo: An open-source generalist robot policy. In Conference on Robot Learning (CoRL), 2024

    2024

  73. [73]

    Open x-embodiment: Robotic learning datasets and rt-x models

    Open X-Embodiment Collaboration. Open x-embodiment: Robotic learning datasets and rt-x models. InIEEE International Conference on Robotics and Automation (ICRA), 2024

    2024

  74. [74]

    Video generation models as world simulators.OpenAI Technical Report, 2024

    OpenAI. Video generation models as world simulators.OpenAI Technical Report, 2024

    2024

  75. [75]

    Genie 2: A large-scale foundation world model

    Jack Parker-Holder, Philip Ball, Jake Bruce, Vibhavari Dasagi, Kristian Holsheimer, Christos Kaplanis, Alexandre Moufarek, Guy Scully, Jeremy Shar, Jimmy Shi, Stephen Spencer, Jessica Yung, Michael Dennis, Sultan Kenjeyev, Shangbang Long, et al. Genie 2: A large-scale foundation world model. https://deepmind.google/discover/blog/ genie-2-a-large-scale-fou...

    2024

  76. [76]

    π0.7: a steerable generalist robotic foundation model with emergent capabilities.arXiv preprint arXiv:2604.15483, 2026

    Physical Intelligence. π0.7: a steerable generalist robotic foundation model with emergent capabilities.arXiv preprint arXiv:2604.15483, 2026

    Pith/arXiv arXiv 2026

  77. [77]

    Physical Intelligence, Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Manuel Y . Galliker, Dibya Ghosh, Lachy Groom, Karol Hausman, Brian Ichter, Szymon Jakubczak, Tim Jones, Liyiming Ke, Devin LeBlanc, Sergey Levine, Adrian Li-Bell, Mohith Mothukuri, Suraj Nair, Karl Pertsc...

    2025

  78. [78]

    Mv-umi: A scalable multi-view interface for cross-embodiment learning.arXiv preprint arXiv:2509.18757, 2025

    Omar Rayyan, John Abanes, Mahmoud Hafez, Anthony Tzes, and Fares Abu-Dakka. Mv-umi: A scalable multi-view interface for cross-embodiment learning.arXiv preprint arXiv:2509.18757, 2025

    arXiv 2025

  79. [79]

    Advancing open-source world models.arXiv preprint arXiv:2601.20540, 2026

    Robbyant Team, Zelin Gao, Qiuyu Wang, Yanhong Zeng, Jiapeng Zhu, Ka Leong Cheng, et al. Advancing open-source world models.arXiv preprint arXiv:2601.20540, 2026

    Pith/arXiv arXiv 2026

  80. [80]

    Learning to act without actions

    Dominik Schmidt and Minqi Jiang. Learning to act without actions. InInt. Conf. Learn. Represent., 2024

    2024

Showing first 80 references.