Pith. sign in

REVIEW 3 major objections 7 minor 133 references

Reviewed by Pith at T0; open to challenge.

T0 means a machine referee read the full paper against a public rubric. The mark states how deep the mechanical check went, never who wrote it. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · glm-5.2

Native video-action pretraining lets robots learn control from web video

2026-07-10 03:52 UTC pith:RNHARHC7

load-bearing objection Real-world claims lack statistical support; simulation gains over prior generation are marginal. But the system integration is genuine and the engineering is serious. the 3 major comments →

arxiv 2607.08639 v1 pith:RNHARHC7 submitted 2026-07-09 cs.RO cs.CV

Native Video-Action Pretraining for Generalizable Robot Control

Qihang Zhang , Lin Li , Luyao Zhang , Shuai Yang , Yiming Luo , Shuaiting Li , Ruilin Wang , Junke Wang
show 21 more authors
Jiahao Shao Gangwei Xu Jiaming Zhou Yishu Shen Yudong Jin Fangyi Xu Shuailei Ma Jiaqi Liao Guanxing Lu Zifan Shi Yongkun Wen Yujie Zhao Weixuan Tang Xinyang Wang Chaojian Li Jiapeng Zhu Ka Leong Cheng Nan Xue Xing Zhu Yujun Shen Yinghao Xu
This is my paper
classification cs.RO cs.CV
keywords modelcontrollingbot-vavideo-actionactioncausaldynamicsfoundation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that robot control models should not be built by bolting an action module onto a video generator designed for digital content creation. Such generators use reconstruction-oriented tokenizers and bidirectional attention, both of which mismatch the demands of physical control: the representation preserves appearance rather than dynamics, and the architecture sees the future during training but cannot at deployment. Instead, the authors pretrain the entire stack from scratch for embodiment. The central object is a semantic visual-action tokenizer that aligns visual latents to a frozen foundation model's features and simultaneously learns compact latent actions through self-supervised inverse and forward dynamics on unlabeled video. This places world states and actions in a single shared latent space, so even web video without action labels provides control-relevant supervision. On top of this space, a causal diffusion transformer with a sparse mixture-of-experts video stream is pretrained natively, learning to predict future visual latents and the latent actions connecting them. The paper claims this native route yields a representation that generalizes from 10-15 demonstrations, transfers across embodiments, and in some cases executes new tasks zero-shot, while an asynchronous inference scheme called Foresight Reasoning enables real-time closed-loop control at up to 225 Hz on physical robots.

Core claim

The paper's central claim is that pretraining a video-action model natively for embodiment—using a semantic visual-action tokenizer that jointly learns visual latents and latent actions from unlabeled web video, combined with a causal architecture trained from scratch—produces a control representation that generalizes better and faster than repurposing generic video generators. The semantic tokenizer is the load-bearing piece: by aligning visual latents to foundation-model semantics and extracting latent actions through self-supervised transition prediction, it converts passive web video into supervision for physical control without requiring robot demonstrations at pretraining scale. The ab

What carries the argument

The semantic visual-action tokenizer (Sec. 2.2) is the central mechanism. It augments a ViT autoencoder with two objectives: (1) semantic alignment, which pulls visual latents toward a frozen Perception Encoder's features via temporal average pooling (Eq. 10), and (2) latent action tokenization, which trains an inverse dynamics model q_phi to predict a compact transition variable l_t between consecutive latents (Eq. 12) and a forward dynamics model f_psi to reconstruct the next latent via a transport map and residual (Eq. 13). The resulting paired visual-action latents serve as training targets for a causal DiT with a sparse MoE video stream (128 routed experts, top-8 routing, loss-free load

Load-bearing premise

The entire architecture rests on the semantic visual-action tokenizer: if the latent actions extracted by self-supervised inverse and forward dynamics on passive web video do not capture control-relevant structure for physical manipulation, then the shared latent space that the causal DiT and downstream policy depend on is misaligned from the start.

What would settle it

Train the semantic visual-action tokenizer on web video alone, then freeze it and train the downstream policy on real-robot tasks. If the resulting policy fails to generalize from 10-15 demonstrations or underperforms a model using a reconstruction-only VAE on real-world manipulation (not just simulation), the claim that self-supervised latent actions from passive video provide control-relevant supervision is not supported.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If self-supervised latent actions from passive web video genuinely capture control-relevant dynamics, then the bottleneck of robot data collection shifts: web-scale video becomes the primary training signal, and robot demonstrations are needed only for fine-tuning rather than for learning dynamics.
  • The causal-from-scratch pretraining route implies that bidirectional video generators pretrained on web data may be the wrong starting point for control, and that the field's common practice of adapting off-the-shelf generators carries a structural tax that native pretraining avoids.
  • The Foresight Reasoning scheme, which predicts future latents in parallel with action execution and re-grounds on real observations, suggests that the serial bottleneck between model inference and robot execution can be broken without losing closed-loop feedback, potentially changing how real-time robot control systems are architected.
  • The human-robot co-training approach, which maps hand poses into a shared action space with robot grippers, implies that egocentric human video can directly improve robot policies during pretraining rather than only during adaptation, if the action-space mismatch is handled at the representation level.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The semantic tokenizer's latent actions are learned from passive video where no physical embodiment is present; whether these abstract transition variables capture the same control structure that a physical gripper's actions impose on the world is an empirical question the paper does not fully resolve, since the tokenizer ablation (Table 2) is conducted only in simulation.
  • If the native pretraining route is correct, then scaling the web-video corpus further should monotonically improve downstream robot performance without additional robot data—a testable prediction that the paper's scaling experiments do not yet push to the regime where this would be the dominant factor.
  • The Foresight Reasoning re-grounding step assumes the learned forward dynamics can correct imagined latents fast enough to prevent drift; the paper does not report how prediction error accumulates over long horizons without real observations, which would determine the scheme's limits in tasks with infrequent visual feedback.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper presents LingBot-VA 2.0, a video-action foundation model for robot manipulation that is pretrained from scratch rather than adapted from generic video generators. The architecture comprises four components: (1) a semantic visual-action tokenizer that aligns visual latents with a frozen foundation model and extracts latent actions via self-supervised inverse/forward dynamics on unlabeled video; (2) a causal diffusion transformer (DiT) with a sparse Mixture-of-Experts (MoE) video stream, trained natively in a causal regime; (3) multi-chunk prediction (MCP), in-context learning (ICL), and human-robot co-training as auxiliary objectives; and (4) Foresight Reasoning, an asynchronous inference scheme that predicts future latents in parallel with action execution and re-grounds on real observations. The system is evaluated on RoboTwin simulation and four real-world manipulation tasks, reporting improvements over π0.5 and the prior LingBot-VA, with a peak asynchronous execution frequency of 225 Hz.

Significance. The paper addresses a well-motivated problem: the mismatch between bidirectional video generators (designed for content creation) and the causal, closed-loop demands of robot control. The native pretraining route—building a semantic tokenizer and causal DiT from scratch rather than retrofitting an off-the-shelf backbone—is a coherent response to this mismatch. The engineering scope is substantial: a 15.3B-parameter model with sparse MoE, consistency distillation, FP8 TensorRT compilation, and paged KV-cache attention optimization, all contributing to a measurable 6.5× end-to-end speedup (Table 3). The Foresight Reasoning scheme (Sec. 2.3.7), which overlaps prediction with execution while re-grounding on real observations, is a concrete and falsifiable contribution to closed-loop control latency. The in-context learning demonstration (Fig. 9) and the human-robot co-training recipe (Sec. 2.3.5) add practical value. However, the central empirical claim—that native pretraining outperforms adapting generic video generators—rests on evaluation evidence that is not yet sufficient in rigor or scale, as detailed below.

major comments (3)
  1. §4.2.1, Figure 8: The real-world evaluation reports success rates and task progress for four tasks but provides no number of evaluation rollouts, no confidence intervals, and no statistical significance tests. Several reported differences between VA 2.0 and VA 1.0 are small (e.g., Pen Collection success rate: 77% vs. 75%; Plate Handover progress: 90% vs. 89%). Without variance estimates or episode counts, it is impossible to determine whether these differences reflect genuine improvement or sampling noise. This is load-bearing because the paper's central claim is that native pretraining outperforms the prior adapted approach. At minimum, the number of rollouts per task and bootstrap or binomial confidence intervals should be reported for all conditions in Figure 8.
  2. Table 1: The simulation improvement over LingBot-VA (the most direct comparison for the native-vs-adapted thesis) is 93.6% vs. 92.2% average, a 1.4 percentage-point difference. The paper does not report the number of evaluation episodes or variance for RoboTwin. A 1.4 pp difference on a benchmark with 50 tasks could be within noise. The paper should either (a) report per-task variance and episode counts to establish that this difference is statistically meaningful, or (b) acknowledge that the simulation gain over the prior generation is marginal and reframe the contribution accordingly. The claim of 'substantial margin' improvement (end of Sec. 1) is not supported by this table alone.
  3. §4.4, Table 2: The tokenizer ablation—the most direct test of the semantic visual-action tokenizer, which is the foundation of the entire architecture—is conducted only in simulation (RoboTwin) and only on a 1.3B model, not the 15.3B deployed system. The MCP ablation (Fig. 10) uses a 5B model. The few-shot and real-world generalization claims (Sec. 4.2) are made for the full-scale system in the real-world domain, but no ablation bridges the gap between the ablated small-model/simulation setting and the full-scale/real-world setting. The paper should either run the tokenizer ablation at or near the deployed scale, or on at least one real-world task, or explicitly discuss the risk that the tokenizer's benefit does not transfer to the full-scale real-world setting.
minor comments (7)
  1. §2.2.2, Eq. (13): The transport map K_t is described as moving information across spatial latent tokens, but its dimensionality and parameterization are not specified. Is K_t a full attention matrix, a per-token linear map, or something else? A brief clarification would help reproducibility.
  2. §2.3.2, Eq. (19): The load-balancing bias update uses a sign function on (c_i - c̄), which zeroes out the mean-centered component. The choice of η_lb and its interaction with training stability is not discussed. A sentence on the range of η_lb used and whether it was tuned would help.
  3. §2.3.7, Eq. (29): The notation v^vid_θ is used both for the video expert's velocity field (Eq. 6) and for the integrated latent obtained from it (Eq. 8, with the noted abuse of notation). In Eq. (29), it is unclear whether FDM_θ denotes the velocity field or the integrated latent. The authors should clarify which quantity is meant, as the reader may confuse the two.
  4. §4.1.1: The model has approximately 15.3B trained parameters with 2.5B active per token. The ratio of MCP heads (1.7B) to the total is notable. It would help to state whether MCP heads are included in the 15.3B count and whether they are discarded at inference (stated in Sec. 2.3.3 but not cross-referenced in the parameter table).
  5. Figure 8: The y-axis labels ('Success Rate' and 'Progress Rate') appear twice with different ranges, and the legend is small. Consider consolidating into a single panel per metric with clearer labeling.
  6. §2.3.5: The hand-to-gripper mapping φ is described qualitatively but the quantile normalization per dataset is not formalized. A brief equation or reference to the normalization procedure would improve reproducibility.
  7. The paper builds on several closely related prior works by the same group (LingBot-VA [50], RepWAM [98], Next Forcing [111], Zero-WAM [60], LingBot-Video [70]). While each is cited, the paper would benefit from a clearer statement of which components are novel vs. inherited, to help readers assess the incremental contribution.

Circularity Check

0 steps flagged

Multiple self-citations for methodological components, but none are load-bearing for the central claim in a circular way; the thesis is tested through experiments against external baselines.

full rationale

The paper builds on several prior works by overlapping authors: the semantic visual-action tokenizer follows RepWAM [98] (Sec. 2.2), multi-chunk prediction follows Next Forcing [111] (Sec. 2.3.3), and in-context learning follows Zero-WAM [60] (Sec. 2.3.4). However, these are methodological adoptions, not circular derivations. Each component is described in sufficient technical detail within this paper (with explicit equations: Eq. 9–14 for the tokenizer, Eq. 21–23 for MCP, Eq. 24 for ICL) and is independently ablated (Table 2 for the tokenizer, Figure 10 for MCP). The central thesis — that native from-scratch causal video-action pretraining outperforms adapting generic bidirectional video generators — is tested through direct experimental comparison against external baselines (π0.5 [77]) and the prior generation (LingBot-VA [50]) in both simulation (Table 1) and real-world deployment (Figure 8). No 'prediction' or 'result' reduces by construction to its inputs. The latent action tokenizer (Eq. 12–14) defines latent actions as compact transition variables learned via self-supervised forward/backward consistency — this is a standard autoencoder-style objective, not a self-definitional loop. The Foresight Reasoning scheme (Eq. 29–30) trains the video expert as a forward-dynamics predictor with an explicit loss; it does not define the prediction as the input. The self-citations are normal methodological lineage, not a chain where the central claim is justified only by citing the authors' own unverified prior work. The derivation is self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

7 free parameters · 4 axioms · 3 invented entities

The paper introduces several design choices as free parameters without always specifying values, and relies on domain assumptions about latent action transfer and causal pretraining that are not fully ablated. The invented entities (tokenizer, Foresight Reasoning, latent actions) have partial independent evidence through simulation ablations but limited isolation on real-world tasks.

free parameters (7)
  • λ_align = Not specified
    Controls strength of semantic alignment loss in visual tokenizer (Eq. 11); value not stated in paper.
  • λ_act = Not specified
    Weight of action loss in video-action objective L_VA = L_vid + λ_act * L_act (Sec. 2.1.2); value not stated.
  • MCP loss weights (w1, w2, w3) = (0.5, 0.2, 0.1)
    Weights for multi-chunk prediction horizons (Eq. 23), adopted from Next Forcing [111].
  • Task mixing distribution π(τ) = Coarse-to-fine schedule
    Controls relative weight of T2I, T2V, TI2VA, ICL, HCT tasks during training (Eq. 28); schedule described qualitatively but not parameterized.
  • MoE routing: Ne, k, γ = Ne=128, k=8, γ not specified
    Number of experts, top-k routing, and scaling factor for sparse MoE (Sec. 2.3.2).
  • Context noise augmentation probability = 0.5
    Probability of replacing clean history frames with noised versions during training (Sec. 4.1.1).
  • Timestep shifts = image=2, web/video=2, robot=5, MCP=10, action=1
    Modality-specific timestep shift parameters for flow-matching training (Sec. 4.1.2).
axioms (4)
  • domain assumption Latent actions extracted from passive video via inverse/forward dynamics capture control-relevant information transferable to robot manipulation.
    Foundational to the entire approach (Sec. 2.2.2); the tokenizer learns latent actions from unlabeled video and the downstream policy operates on these latents. Not independently verified on real-world tasks.
  • domain assumption Causal pretraining from scratch avoids catastrophic forgetting better than bidirectional-to-causal finetuning.
    Stated in Sec. 1 and Sec. 2.3.6 as motivation for from-scratch training. No direct ablation comparing from-scratch causal vs. bidirectional-pretrained-then-causal-finetuned is provided.
  • domain assumption The frozen Perception Encoder [10] provides semantic features suitable for aligning robot control representations.
    Used as teacher model G in Eq. 10 for semantic alignment of the visual tokenizer.
  • domain assumption Human hand motion retargeted to gripper apertures provides useful action supervision for robot policies.
    Underlies the human-robot co-training of Sec. 2.3.5; the functional gripper opening map φ is assumed to transfer meaningful grasp timing.
invented entities (3)
  • Semantic visual-action tokenizer independent evidence
    purpose: Jointly encodes visual states and latent actions in a shared semantic latent space aligned to a foundation model
    Ablated against reconstruction VAE in Table 2 on RoboTwin simulation, showing improvement. Not ablated on real-world tasks.
  • Foresight Reasoning no independent evidence
    purpose: Asynchronous inference scheme that predicts future latents in parallel with execution and re-grounds on real observations
    Described in Sec. 2.3.7 with forward-dynamics grounding loss (Eq. 30). No ablation isolating Foresight Reasoning from the rest of the system is provided; the 225 Hz claim includes all acceleration techniques (Table 3) but does not separate the contribution of asynchronous re-grounding.
  • Latent action token (a_t ≡ ℓ_t) independent evidence
    purpose: Compact transition variable between visual latents serving as action representation
    Learned via self-supervised inverse/forward dynamics (Eq. 12-14) with bottleneck. The approach follows RepWAM [98] and is validated indirectly through downstream task performance.

pith-pipeline@v1.1.0-glm · 34301 in / 3489 out tokens · 599421 ms · 2026-07-10T03:52:24.423418+00:00 · methodology

0 comments
read the original abstract

The advent of video-action models offers a promising path for robot control. Nevertheless, we argue that repurposing video generative models designed for digital content creation is inherently inadequate for physical environments. To bridge this gap, we present LingBot-VA 2.0, a video-action foundation model built from the ground up for embodiment. Four core design principles showcase its evolution from LingBot-VA. (1) Departing from traditional reconstruction-focused VAEs, we introduce a semantic visual-action tokenizer, which aligns visual representations with both semantics and actions, improving instruction following and action precision in subsequent policy learning. (2) Given the strictly causal nature of temporal dynamics, we adopt a causal pretraining paradigm, training from scratch to circumvent the catastrophic forgetting that frequently occurs when adapting bidirectional architectures. (3) To meet the demands of high-frequency inference, our model employs a sparse MoE backbone, expanding model capacity without compromising efficiency. (4) Real-time closed-loop control is realized through an enhanced asynchronous inference scheme, which predicts future latents in parallel with action execution while re-grounding each rollout on the latest observation via learned forward dynamics. Real-world deployment validates LingBot-VA 2.0 as a robust foundation model, as evidenced by its few-shot generalization across complex manipulation tasks.

Figures

Figures reproduced from arXiv: 2607.08639 by Chaojian Li, Fangyi Xu, Gangwei Xu, Guanxing Lu, Jiahao Shao, Jiaming Zhou, Jiapeng Zhu, Jiaqi Liao, Junke Wang, Ka Leong Cheng, Lin Li, Luyao Zhang, Nan Xue, Qihang Zhang, Ruilin Wang, Shuailei Ma, Shuaiting Li, Shuai Yang, Weixuan Tang, Xing Zhu, Xinyang Wang, Yiming Luo, Yinghao Xu, Yishu Shen, Yongkun Wen, Yudong Jin, Yujie Zhao, Yujun Shen, Zifan Shi.

Figure 1
Figure 1. Figure 1: Overview of LingBot-VA 2.0. The Planner (Sec. 2.3.1) decomposes long-horizon goals into structured subtask context. The semantic visual-action tokenizer (Sec. 2.2) encodes image observations and human video prompts (Sec. 2.3.4) as visual latents. The video model with the MoE (Sec. 2.3.2) and the action model autoregressively predict future visual latents and robot actions. MCP (Sec. 2.3.3) adds future-chun… view at source ↗
Figure 2
Figure 2. Figure 2: Semantic visual-action tokenizer. The visual tokenizer aligns reconstruction-oriented latents with visual foundation￾model features, while the latent action tokenizer learns compact transition variables through inverse and forward dynamics. Adapted from RepWAM [98]. 2.2.1 Visual Tokenization with Semantic Alignment We adopt a Vision Transformer (ViT) autoencoder for visual tokenization [25, 97]. Given a vi… view at source ↗
Figure 3
Figure 3. Figure 3: Dual-system hierarchical policy. A high-level VLM planner consumes the task goal and sparse recent observations to produce structured subtask context, which conditions the low-level video-action policy together with robot state to produce closed-loop actions. where (Kt, δt) are predicted from ℓt. The transport Kt moves information across spatial latent tokens, while δt accounts for changes that cannot be e… view at source ↗
Figure 4
Figure 4. Figure 4: Training loss comparison between the MoE-13B-A1.9B video-stream model and the Dense-5B baseline. Left: loss aligned by optimization steps, where both models remain close and the dense baseline is slightly lower. Right: loss aligned by wall-clock training time, where the final losses nearly coincide. where σ(·) is the sigmoid function and Ne is the number of routed experts. We use group-limited top-k routin… view at source ↗
Figure 5
Figure 5. Figure 5: Multi-Chunk Prediction (MCP). Lightweight auxiliary modules predict several future latent chunks at increasing horizons from the main model’s representation, providing dense temporal supervision during training. Adapted from Next Forcing [111]. chunk, creating a myopic supervision problem in which the learning signal becomes overly local: adjacent chunks, especially at high frame rates, are often visually … view at source ↗
Figure 6
Figure 6. Figure 6: Foresight Reasoning as an asynchronous inference/execution flow. A cold start initializes the KV cache and pre￾computes the first action chunk. During the loop, the executor runs the current chunk while real observations enter OBSQUEUE; the inference branch re-grounds the KV cache with those observations, appends the currently executing action, imagines its visual outcome with FDM grounding, and predicts t… view at source ↗
Figure 7
Figure 7. Figure 7: In-context human-robot data curation pipeline. Robot videos are sampled by task semantics, converted into human demonstration prompts with a VLM, synthesized as human videos, and filtered before pairing with the original robot trajectories as ICL samples. dataset encompasses data from over 10 robot video pre-training datasets (e.g., AgiBot [2], OXE [73]), comprising more than 5,000 tasks and over 50,000 sa… view at source ↗
Figure 8
Figure 8. Figure 8: Real-world deployment results. Success rate and task progress rate across real-world tasks, comparing LingBot-VA 2.0 against representative baselines. task-level context across multiple interaction steps, while its video grounding also improves the precision of local manipulation. Compared with LingBot-VA, the gains come from causal video-action pretraining, which exposes the model to more generalizable vi… view at source ↗
Figure 9
Figure 9. Figure 9: In-context learning demo. We demonstrate ICL rollouts on four unseen tasks. For each rollout, given a reference video demonstration as the task instruction and the robot observation, LingBot-VA 2.0 transfers the same action procedure to robot manipulation without parameter updates [PITH_FULL_IMAGE:figures/full_fig_p019_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Multi-chunk prediction ablation. Task success rate (%) on RoboTwin across training steps. MCP converges faster and reaches higher final accuracy than the baseline at both 12 and 50 fps. and optimization setup, with MCP enabled in only one of the two variants. MCP consistently improves optimization efficiency, converging faster and reaching higher final task success at both 12 and 50 fps. The advantage is … view at source ↗
Figure 11
Figure 11. Figure 11: Real-world demonstrations. Qualitative rollouts of LingBot-VA 2.0 across four representative tasks, covering long-horizon tabletop organization, interaction with moving objects, fine-grained grasping, and reactive visual control. 5 Related Work 5.1 Generalist robot policies and VLA models Vision-language-action (VLA) models directly map visual observations and language instructions to actions, inheriting … view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

133 extracted references · 133 canonical work pages · 53 internal anchors

  1. [1]

    1x world model: From video to action

    1X Technologies. 1x world model: From video to action. https://www.1x.tech/discover/world-model-self-learning,

    work page

  2. [2]

    AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systems

    AgiBot-World-Contributors, Qingwen Bu, Jisong Cai, Li Chen, Xiuqi Cui, Yan Ding, Siyuan Feng, Shenyuan Gao, Xindong He, Xuan Hu, Xu Huang, Shu Jiang, Yuxin Jiang, Cheng Jing, Hongyang Li, et al. Agibot world colosseo: A large-scale manipulation platform for scalable and intelligent embodied systems.arXiv preprint arXiv:2503.06669, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  3. [3]

    Video pretraining (vpt): Learning to act by watching unlabeled online videos

    Bowen Baker, Ilge Akkaya, Peter Zhokov, Joost Huizinga, Jie Tang, Adrien Ecoffet, Brandon Houghton, Raul Sampedro, and Jeff Clune. Video pretraining (vpt): Learning to act by watching unlabeled online videos. InAdv. Neural Inform. Process. Syst., 2022

    work page 2022

  4. [4]

    One transformer fits all distributions in multi-modal diffusion at scale

    Fan Bao, Shen Nie, Kaiwen Xue, Chongxuan Li, Shi Pu, Yaole Wang, Gang Yue, Yue Cao, Hang Su, and Jun Zhu. One transformer fits all distributions in multi-modal diffusion at scale. InInt. Conf. Mach. Learn., 2023

    work page 2023

  5. [5]

    Navigation world models

    Amir Bar, Gaoyue Zhou, Danny Tran, Trevor Darrell, and Yann LeCun. Navigation world models. InIEEE Conf. Comput. Vis. Pattern Recog., 2025

    work page 2025

  6. [6]

    A Careful Examination of Large Behavior Models for Multitask Dexterous Manipulation

    Jose Barreiros, Andrew Beaulieu, Aditya Bhat, Rick Cory, Eric Cousineau, Hongkai Dai, Ching-Hsin Fang, Kunimatsu Hashimoto, Muhammad Zubair Irshad, Masha Itkina, et al. A careful examination of large behavior models for multitask dexterous manipulation.arXiv preprint arXiv:2507.05331, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  7. [7]

    Gen2act: Human video generation in novel scenarios enables generalizable robot manipulation

    Homanga Bharadhwaj, Debidatta Dwibedi, Abhinav Gupta, Shubham Tulsiani, Carl Doersch, Ted Xiao, Dhruv Shah, Fei Xia, Dorsa Sadigh, and Sean Kirmani. Gen2act: Human video generation in novel scenarios enables generalizable robot manipulation. InConference on Robot Learning (CoRL), 2024

    work page 2024

  8. [8]

    Motus: A Unified Latent Action World Model

    Hongzhe Bi, Hengkai Tan, Shenghao Xie, Zeyuan Wang, Shuhe Huang, Haitian Liu, Ruowen Zhao, Yao Feng, Chendong Xiang, Yinze Rong, Hongyan Zhao, Hanyu Liu, Zhizhong Su, Lei Ma, Hang Su, et al. Motus: A unified latent action world model.arXiv preprint arXiv:2512.13030, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  9. [9]

    π0: A vision- language-action flow model for general robot control

    Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, Szymon Jakubczak, Tim Jones, Liyiming Ke, Sergey Levine, Adrian Li-Bell, et al. π0: A vision- language-action flow model for general robot control. InRobotics: Science and Systems, 2025

    work page 2025

  10. [10]

    Perception encoder: The best visual embeddings are not at the output of the network

    Daniel Bolya, Po-Yao Huang, Peize Sun, Jang Hyun Cho, Andrea Madotto, Chen Wei, Tengyu Ma, Jiale Zhi, Jathushan Rajasegaran, Hanoona Bangalath, et al. Perception encoder: The best visual embeddings are not at the output of the network. InNeurIPS, 2025

    work page 2025

  11. [11]

    Rt-2: Vision-language-action models transfer web knowledge to robotic control

    Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choromanski, Tianli Ding, Danny Driess, Avinava Dubey, Chelsea Finn, Pete Florence, Chuyuan Fu, Montse Gonzalez Arenas, Keerthana Gopalakrishnan, Kehang Han, et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control. InConference on Robot Learning ...

    work page 2023

  12. [12]

    Rt-1: Robotics transformer for real-world control at scale

    Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Jasmine Hsu, Julian Ibarz, Brian Ichter, Alex Irpan, Tomas Jackson, Sally Jesmonth, et al. Rt-1: Robotics transformer for real-world control at scale. InRobotics: Science and Systems, 2023. 23

    work page 2023

  13. [13]

    Genie: Generative interactive environments

    Jake Bruce, Michael Dennis, Ashley Edwards, Jack Parker-Holder, Yuge Shi, Edward Hughes, Matthew Lai, Aditi Mavalankar, Richie Steigerwald, Chris Apps, Yusuf Aytar, Sarah Bechtle, Feryal Behbahani, Stephanie Chan, Nicolas Heess, et al. Genie: Generative interactive environments. InInt. Conf. Mach. Learn., 2024

    work page 2024

  14. [14]

    Univla: Learning to act anywhere with task-centric latent actions

    Qingwen Bu, Yanting Yang, Jisong Cai, Shenyuan Gao, Guanghui Ren, Maoqing Yao, Ping Luo, and Hongyang Li. Univla: Learning to act anywhere with task-centric latent actions. InRobotics: Science and Systems, 2025

    work page 2025

  15. [15]

    WorldVLA: Towards Autoregressive Action World Model

    Jun Cen, Chaohui Yu, Hangjie Yuan, Yuming Jiang, Siteng Huang, Jiayan Guo, Xin Li, Yibing Song, Hao Luo, Fan Wang, Deli Zhao, and Hao Chen. Worldvla: Towards autoregressive action world model.arXiv preprint arXiv:2506.21539, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  16. [16]

    Gamegen-x: Interactive open-world game video generation

    Haoxuan Che, Xuanhua He, Quande Liu, Cheng Jin, and Hao Chen. Gamegen-x: Interactive open-world game video generation. InInt. Conf. Learn. Represent., 2025

    work page 2025

  17. [17]

    GR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulation

    Chi-Lam Cheang, Guangzeng Chen, Ya Jing, Tao Kong, Hang Li, Yifeng Li, Yuxiao Liu, Hongtao Wu, Jiafeng Xu, Yichu Yang, Hanbo Zhang, and Minzhao Zhu. Gr-2: A generative video-language-action model with web-scale knowledge for robot manipulation.arXiv preprint arXiv:2410.06158, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  18. [18]

    Diffusion forcing: Next-token prediction meets full-sequence diffusion

    Boyuan Chen, Diego Marti Monso, Yilun Du, Max Simchowitz, Russ Tedrake, and Vincent Sitzmann. Diffusion forcing: Next-token prediction meets full-sequence diffusion. InAdv. Neural Inform. Process. Syst., 2024

    work page 2024

  19. [19]

    RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation

    Tianxing Chen, Zanxin Chen, Baijun Chen, Zijian Cai, Yibin Liu, Zixuan Li, Qiwei Liang, Xianliang Lin, Yiheng Ge, Zhenyu Gu, Weiliang Deng, Yubin Guo, Tian Nian, Xuanbing Xie, Qiangyu Chen, et al. Robotwin 2.0: A scalable data generator and benchmark with strong domain randomization for robust bimanual robotic manipulation.arXiv preprint arXiv:2506.18088, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  20. [20]

    Moto: Latent motion token as the bridging language for learning robot manipulation from videos

    Yi Chen, Yuying Ge, Weiliang Tang, Yizhuo Li, Yixiao Ge, Mingyu Ding, Ying Shan, and Xihui Liu. Moto: Latent motion token as the bridging language for learning robot manipulation from videos. InInt. Conf. Comput. Vis., 2025

    work page 2025

  21. [21]

    Diffusion policy: Visuomotor policy learning via action diffusion

    Cheng Chi, Siyuan Feng, Yilun Du, Zhenjia Xu, Eric Cousineau, Benjamin Burchfiel, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. InRobotics: Science and Systems, 2023

    work page 2023

  22. [22]

    Universal manipulation interface: In-the-wild robot teaching without in-the-wild robots

    Cheng Chi, Zhenjia Xu, Chuer Pan, Eric Cousineau, Benjamin Burchfiel, Siyuan Feng, Russ Tedrake, and Shuran Song. Universal manipulation interface: In-the-wild robot teaching without in-the-wild robots. InRobotics: Science and Systems, 2024

    work page 2024

  23. [23]

    Damai Dai, Chengqi Deng, Chenggang Zhao, R. X. Xu, Huazuo Gao, Deli Chen, Jiashi Li, Wangding Zeng, Xingkai Yu, Y . Wu, Zhenda Xie, Y . K. Li, Panpan Huang, Fuli Luo, Chong Ruan, Zhifang Sui, and Wenfeng Liang. DeepSeekMoE: Towards ultimate expert specialization in mixture-of-experts language models. InProceedings of the Annual Meeting of the Association ...

    work page 2024

  24. [24]

    DeepSeek-V3 Technical Report

    DeepSeek-AI. DeepSeek-V3 technical report.arXiv preprint arXiv:2412.19437, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  25. [25]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. InInt. Conf. Learn. Represent., 2021

    work page 2021

  26. [26]

    Tenenbaum, Dale Schuurmans, and Pieter Abbeel

    Yilun Du, Mengjiao Yang, Bo Dai, Hanjun Dai, Ofir Nachum, Joshua B. Tenenbaum, Dale Schuurmans, and Pieter Abbeel. Learning universal policies via text-guided video generation. InAdv. Neural Inform. Process. Syst., 2023

    work page 2023

  27. [27]

    Learning universal policies via text-guided video generation.Advances in neural information processing systems, 36:9156–9172, 2023

    Yilun Du, Sherry Yang, Bo Dai, Hanjun Dai, Ofir Nachum, Josh Tenenbaum, Dale Schuurmans, and Pieter Abbeel. Learning universal policies via text-guided video generation.Advances in neural information processing systems, 36:9156–9172, 2023

    work page 2023

  28. [28]

    Adaworld: Learning adaptable world models with latent actions

    Shenyuan Gao, Siyuan Zhou, Yilun Du, Jun Zhang, and Chuang Gan. Adaworld: Learning adaptable world models with latent actions. InInt. Conf. Mach. Learn., 2025

    work page 2025

  29. [29]

    Infinite Worlds with Versatile Interactions

    Zelin Gao, Qiuyu Wang, Jiapeng Zhu, Jingye Chen, Zichen Liu, Qingyan Bai, et al. Infinite worlds with versatile interactions. arXiv preprint arXiv:2607.07534, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  30. [30]

    Gemini Robotics: Bringing AI into the Physical World

    Gemini Robotics Team. Gemini robotics: Bringing ai into the physical world.arXiv preprint arXiv:2503.20020, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  31. [31]

    Gen-0: Embodied foundation models that scale with physical interaction

    Generalist AI. Gen-0: Embodied foundation models that scale with physical interaction. https://generalistai.com/blog/ nov-04-2025-GEN-0, 2025. Built on Harmonic Reasoning

    work page 2025

  32. [32]

    Veo: A text-to-video generation system.Google DeepMind Technical Report, 2025

    Google DeepMind. Veo: A text-to-video generation system.Google DeepMind Technical Report, 2025

    work page 2025

  33. [33]

    Mastering diverse control tasks through world models

    Danijar Hafner, Jurgis Pasukonis, Jimmy Ba, and Timothy Lillicrap. Mastering diverse control tasks through world models. Nature, 2025. 24

    work page 2025

  34. [34]

    Td-mpc2: Scalable, robust world models for continuous control

    Nicklas Hansen, Hao Su, and Xiaolong Wang. Td-mpc2: Scalable, robust world models for continuous control. InInt. Conf. Learn. Represent., 2024

    work page 2024

  35. [35]

    Video prediction policy: A generalist robot policy with predictive visual representations

    Yucheng Hu, Yanjiang Guo, Pengchao Wang, Xiaoyu Chen, Yen-Jen Wang, Jianke Zhang, Koushil Sreenath, Chaochao Lu, and Jianyu Chen. Video prediction policy: A generalist robot policy with predictive visual representations. InInt. Conf. Mach. Learn., 2025

    work page 2025

  36. [36]

    Enerverse: Envisioning embodied future space for robotics manipulation.arXiv preprint arXiv:2501.01895, 2025

    Siyuan Huang, Liliang Chen, Pengfei Liu, Yue Hu, Shengyu Zhang, Peng Gao, Hongsheng Li, Maoqing Yao, and Guanghui Ren. Enerverse: Envisioning embodied future space for robotics manipulation.arXiv preprint arXiv:2501.01895, 2025

    work page arXiv 2025

  37. [37]

    Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion

    Xun Huang, Zhengqi Li, Guande He, Mingyuan Zhou, and Eli Shechtman. Self forcing: Bridging the train-test gap in autoregressive video diffusion.arXiv preprint arXiv:2506.08009, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  38. [38]

    NORA: A Small Open-Sourced Generalist Vision Language Action Model for Embodied Tasks

    Chia-Yu Hung, Qi Sun, Pengfei Hong, Amir Zadeh, Chuan Li, U-Xuan Tan, Navonil Majumder, and Soujanya Poria. Nora: A small open-sourced generalist vision language action model for embodied tasks.arXiv preprint arXiv:2504.19854, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  39. [39]

    Dreamgen: Unlocking generalization in robot learning through video world models

    Joel Jang, Seonghyeon Ye, Zongyu Lin, et al. Dreamgen: Unlocking generalization in robot learning through video world models. InConference on Robot Learning (CoRL), 2025

    work page 2025

  40. [40]

    EgoMimic: Scaling Imitation Learning via Egocentric Video

    Simar Kareer, Dhruv Patel, Ryan Punamiya, Pranay Mathur, Shuo Cheng, Chen Wang, Judy Hoffman, and Danfei Xu. Egomimic: Scaling imitation learning via egocentric video.arXiv preprint arXiv:2410.24221, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  41. [41]

    Droid: A large-scale in-the-wild robot manipulation dataset

    Alexander Khazatsky, Karl Pertsch, Suraj Nair, Ashwin Balakrishna, Sudeep Dasari, Siddharth Karamcheti, Soroush Nasiriany, Mohan Kumar Srirama, Lawrence Yunliang Chen, Kirsty Ellis, Peter David Fagan, Joey Hejna, Masha Itkina, Marion Lepert, Yecheng Jason Ma, et al. Droid: A large-scale in-the-wild robot manipulation dataset. InRobotics: Science and Systems, 2024

    work page 2024

  42. [42]

    Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planning

    Moo Jin Kim, Yihuai Gao, Tsung-Yi Lin, Yen-Chen Lin, Yunhao Ge, Grace Lam, Percy Liang, Shuran Song, Ming-Yu Liu, Chelsea Finn, et al. Cosmos policy: Fine-tuning video models for visuomotor control and planning.arXiv preprint arXiv:2601.16163, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  43. [43]

    Openvla: An open-source vision-language-action model

    Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, Quan Vuong, Thomas Kollar, Benjamin Burchfiel, Russ Tedrake, Dorsa Sadigh, et al. Openvla: An open-source vision-language-action model. InConference on Robot Learning (CoRL), 2024

    work page 2024

  44. [44]

    Auto-Encoding Variational Bayes

    Diederik P Kingma and Max Welling. Auto-encoding variational bayes.arXiv preprint arXiv:1312.6114, 2013

    work page internal anchor Pith review Pith/arXiv arXiv 2013

  45. [45]

    Kling-v3.https://kling.ai/, 2026

    KlingAI Team. Kling-v3.https://kling.ai/, 2026

    work page 2026

  46. [46]

    Tenenbaum

    Po-Chen Ko, Jiayuan Mao, Yilun Du, Shao-Hua Sun, and Joshua B. Tenenbaum. Learning to act from actionless videos through dense correspondences. InInt. Conf. Learn. Represent., 2024

    work page 2024

  47. [47]

    Deformnet: Latent space modeling and dynamics prediction for deformable object manipulation

    Chenchang Li, Zihao Ai, Tong Wu, Xiaosa Li, Wenbo Ding, and Huazhe Xu. Deformnet: Latent space modeling and dynamics prediction for deformable object manipulation. In2024 IEEE International Conference on Robotics and Automation (ICRA), pages 14770–14776. IEEE, 2024

    work page 2024

  48. [48]

    Cronusvla: Transferring latent motion across time for multi-frame prediction in manipulation,

    Hao Li, Shuai Yang, Yilun Chen, Yang Tian, Xiaoda Yang, Xinyi Chen, Hanqing Wang, Tai Wang, Feng Zhao, Dahua Lin, et al. Cronusvla: Transferring latent motion across time for multi-frame prediction in manipulation.arXiv preprint arXiv:2506.19816, 2025

    work page arXiv 2025

  49. [49]

    GR-3 Technical Report

    Jiacheng Li, Mengzhou Sun, Bowen Zhang, Zhe Zhao, Xiu Liu, et al. Gr-3 technical report.arXiv preprint arXiv:2507.15493, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  50. [50]

    Causal World Modeling for Robot Control

    Lin Li, Qihang Zhang, Yiming Luo, Shuai Yang, Ruilin Wang, Fei Han, Mingrui Yu, Zelin Gao, Nan Xue, Xing Zhu, Yujun Shen, and Yinghao Xu. Causal world modeling for robot control.arXiv preprint arXiv:2601.21998, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  51. [51]

    CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulation

    Qixiu Li, Yaobo Liang, Zeyu Wang, Lin Luo, Xi Chen, Mozheng Liao, Fangyun Wei, Yu Deng, Sicheng Xu, Yizhong Zhang, Xiaofan Wang, Bei Liu, Jianlong Fu, Jianmin Bao, Dong Chen, Yuanchun Shi, Jiaolong Yang, and Baining Guo. Cogact: A foundational vision-language-action model for synergizing cognition and action in robotic manipulation.arXiv preprint arXiv:24...

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  52. [52]

    What Matters When Cotraining Robot Manipulation Policies on Everyday Human Videos?

    Richard Li, Aditya Prakash, Andrew Wen, Saurabh Gupta, Yilun Du, and Pulkit Agrawal. What matters when cotraining robot manipulation policies on everyday human videos?arXiv preprint arXiv:2606.06627, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  53. [53]

    WALL-WM: Carving World Action Modeling at the Event Joints

    Shalfun Li, Victor Yao, Charles Yang, Truth Qu, Regis Cheng, Ryan Yu, Howard Lu, Newton V on, Vincent Chen, Yohann Tang, et al. Wall-wm: Carving world action modeling at the event joints.arXiv preprint arXiv:2606.01955, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  54. [54]

    Unified video action model

    Shuang Li, Yihuai Gao, Dorsa Sadigh, and Shuran Song. Unified video action model. InRobotics: Science and Systems, 2025. 25

    work page 2025

  55. [55]

    Vision-language foundation models as effective robot imitators

    Xinghang Li, Minghuan Liu, Hanbo Zhang, Cunjun Yu, Jie Xu, Hongtao Wu, Chilam Cheang, Ya Jing, Weinan Zhang, Huaping Liu, Hang Li, and Tao Kong. Vision-language foundation models as effective robot imitators. InInt. Conf. Learn. Represent., 2024

    work page 2024

  56. [56]

    Propagation networks for model-based control under partial observation

    Yunzhu Li, Jiajun Wu, Jun-Yan Zhu, Joshua B Tenenbaum, Antonio Torralba, and Russ Tedrake. Propagation networks for model-based control under partial observation. In2019 International Conference on Robotics and Automation (ICRA), pages 1205–1211. IEEE, 2019

    work page 2019

  57. [57]

    Dreamitate: Real-world visuomotor policy learning via video generation

    Junbang Liang, Ruoshi Liu, Ege Ozguroglu, Sruthi Sudhakar, Achal Dave, Pavel Tokmakov, Shuran Song, and Carl V ondrick. Dreamitate: Real-world visuomotor policy learning via video generation. InConference on Robot Learning (CoRL), 2024

    work page 2024

  58. [58]

    Mixture-of-transformers: A sparse and scalable architecture for multi-modal foundation models.Transactions on Machine Learning Research, 2025

    Weixin Liang, LILI YU, Liang Luo, Srini Iyer, Ning Dong, Chunting Zhou, Gargi Ghosh, Mike Lewis, Wen tau Yih, Luke Zettlemoyer, and Xi Victoria Lin. Mixture-of-transformers: A sparse and scalable architecture for multi-modal foundation models.Transactions on Machine Learning Research, 2025

    work page 2025

  59. [59]

    Genie Envisioner: A Unified World Foundation Platform for Robotic Manipulation

    Yue Liao, Pengfei Zhou, Siyuan Huang, Donglin Yang, Shengcong Chen, Yuxin Jiang, Yue Hu, Jingbin Cai, Si Liu, Jianlan Luo, Liliang Chen, Shuicheng Yan, Maoqing Yao, and Guanghui Ren. Genie envisioner: A unified world foundation platform for robotic manipulation.arXiv preprint arXiv:2508.05635, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  60. [60]

    Zero-wam: In-context world modeling for zero-shot task generalization

    LingBot-V A Team, RobbyAnt. Zero-wam: In-context world modeling for zero-shot task generalization. https://github. com/jiaming-zhou/Zero-W AM, 2026

    work page 2026

  61. [61]

    Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. InInt. Conf. Learn. Represent., 2023

    work page 2023

  62. [62]

    Efficient robotic policy learning via latent space backward planning

    Dongxiu Liu, Haoyi Niu, Zhihao Wang, Jinliang Zheng, Yinan Zheng, Zhonghong Ou, Jianming Hu, Jianxiong Li, and Xianyuan Zhan. Efficient robotic policy learning via latent space backward planning. InInt. Conf. Mach. Learn., 2025

    work page 2025

  63. [63]

    HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Model

    Jiaming Liu, Hao Chen, Pengju An, Zhuoyang Liu, Renrui Zhang, Chenyang Gu, Xiaoqi Li, Ziyu Guo, Sixiang Chen, Mengzhen Liu, Chengkai Hou, Mengdi Zhao, KC alex Zhou, Pheng-Ann Heng, and Shanghang Zhang. Hybridvla: Collaborative diffusion and autoregression in a unified vision-language-action model.arXiv preprint arXiv:2503.10631, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  64. [64]

    Muon is Scalable for LLM Training

    Jingyuan Liu, Jianlin Su, Xingcheng Yao, Zhejun Jiang, Guokun Lai, Yulun Du, Yidao Qin, Weixin Xu, Enzhe Lu, Junjie Yan, Yanru Chen, Huabin Zheng, Yibo Liu, Shaowei Liu, Bohong Yin, Weiran He, Han Zhu, Yuzhi Wang, Jianzhou Wang, Mengnan Dong, Zheng Zhang, Yongsheng Kang, Hao Zhang, Xinran Xu, Yutao Zhang, Yuxin Wu, Xinyu Zhou, and Zhilin Yang. Muon is sca...

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  65. [65]

    Rdt-1b: a diffusion foundation model for bimanual manipulation

    Songming Liu, Lingxuan Wu, Bangguo Li, Hengkai Tan, Huayu Chen, Zhengyi Wang, Ke Xu, Hang Su, and Jun Zhu. Rdt-1b: a diffusion foundation model for bimanual manipulation. InInt. Conf. Learn. Represent., 2025

    work page 2025

  66. [66]

    Being-H0: Vision-Language-Action Pretraining from Large-Scale Human Videos

    Hao Luo, Yicheng Feng, Wanpeng Zhang, Sipeng Zheng, Ye Wang, Haoqi Yuan, Jiazheng Liu, Chaoyi Xu, Qin Jin, and Zongqing Lu. Being-h0: Vision-language-action pretraining from large-scale human videos.arXiv preprint arXiv:2507.15597, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  67. [67]

    Being-H0.7: A Latent World-Action Model from Egocentric Videos

    Hao Luo, Wanpeng Zhang, Yicheng Feng, Sipeng Zheng, Haiweng Xu, Chaoyi Xu, Ziheng Xi, Yuhui Fu, and Zongqing Lu. Being-h0.7: A latent world-action model from egocentric videos.arXiv preprint arXiv:2605.00078, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  68. [68]

    Deep learning for universal linear embeddings of nonlinear dynamics

    Bethany Lusch, J Nathan Kutz, and Steven L Brunton. Deep learning for universal linear embeddings of nonlinear dynamics. Nature communications, 9(1):4950, 2018

    work page 2018

  69. [69]

    F1: A Vision-Language-Action Model Bridging Understanding and Generation to Actions

    Qi Lv, Weijie Kong, Hao Li, Jia Zeng, Zherui Qiu, Delin Qu, Haoming Song, Qizhi Chen, Xiang Deng, and Jiangmiao Pang. F1: A vision-language-action model bridging understanding and generation to actions.arXiv preprint arXiv:2509.06951, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  70. [70]

    Scaling Mixture-of-Experts Video Pretraining for Embodied Intelligence

    Shuailei Ma, Jiaqi Liao, Xinyang Wang, Jingjing Wang, Chaoran Feng, Zijing Hu, Chong Bao, Zichen Xi, Yuqi Gan, Weisen Wang, Yanhong Zeng, Qin Zhao, Zifan Shi, Wei Wu, Hao Ouyang, Qiuyu Wang, Shangzhan Zhang, Jiahao Shao, Yipengjing Sun, Liangxiao Hu, Lunke Pan, Nan Xue, Kecheng Zheng, Yinghao Xu, Xing Zhu, Yujun Shen, and Ka Leong Cheng. Scaling mixture-o...

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  71. [71]

    GR00T N1: An Open Foundation Model for Generalist Humanoid Robots

    NVIDIA. Gr00t n1: An open foundation model for generalist humanoid robots.arXiv preprint arXiv:2503.14734, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  72. [72]

    Octo: An open-source generalist robot policy

    Octo Model Team, Dibya Ghosh, Homer Walke, Karl Pertsch, et al. Octo: An open-source generalist robot policy. In Conference on Robot Learning (CoRL), 2024

    work page 2024

  73. [73]

    Open x-embodiment: Robotic learning datasets and rt-x models

    Open X-Embodiment Collaboration. Open x-embodiment: Robotic learning datasets and rt-x models. InIEEE International Conference on Robotics and Automation (ICRA), 2024

    work page 2024

  74. [74]

    Video generation models as world simulators.OpenAI Technical Report, 2024

    OpenAI. Video generation models as world simulators.OpenAI Technical Report, 2024. 26

    work page 2024

  75. [75]

    Genie 2: A large-scale foundation world model

    Jack Parker-Holder, Philip Ball, Jake Bruce, Vibhavari Dasagi, Kristian Holsheimer, Christos Kaplanis, Alexandre Moufarek, Guy Scully, Jeremy Shar, Jimmy Shi, Stephen Spencer, Jessica Yung, Michael Dennis, Sultan Kenjeyev, Shangbang Long, et al. Genie 2: A large-scale foundation world model. https://deepmind.google/discover/blog/ genie-2-a-large-scale-fou...

    work page 2024

  76. [76]

    ${\pi}_{0.7}$: a Steerable Generalist Robotic Foundation Model with Emergent Capabilities

    Physical Intelligence. π0.7: a steerable generalist robotic foundation model with emergent capabilities.arXiv preprint arXiv:2604.15483, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  77. [77]

    Physical Intelligence, Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Manuel Y . Galliker, Dibya Ghosh, Lachy Groom, Karol Hausman, Brian Ichter, Szymon Jakubczak, Tim Jones, Liyiming Ke, Devin LeBlanc, Sergey Levine, Adrian Li-Bell, Mohith Mothukuri, Suraj Nair, Karl Pertsc...

    work page 2025

  78. [78]

    Mv-umi: A scalable multi-view interface for cross-embodiment learning.arXiv preprint arXiv:2509.18757, 2025

    Omar Rayyan, John Abanes, Mahmoud Hafez, Anthony Tzes, and Fares Abu-Dakka. Mv-umi: A scalable multi-view interface for cross-embodiment learning.arXiv preprint arXiv:2509.18757, 2025

    work page arXiv 2025

  79. [79]

    Advancing Open-source World Models

    Robbyant Team, Zelin Gao, Qiuyu Wang, Yanhong Zeng, Jiapeng Zhu, Ka Leong Cheng, et al. Advancing open-source world models.arXiv preprint arXiv:2601.20540, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  80. [80]

    Learning to act without actions

    Dominik Schmidt and Minqi Jiang. Learning to act without actions. InInt. Conf. Learn. Represent., 2024

    work page 2024

Showing first 80 references.