Pith. sign in

REVIEW 4 major objections 6 minor 73 references

A single 38B foundation model can generate multi-view robot scenes, transfer them controllably, and roll them into videos that improve real robot policies.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-14 04:09 UTC pith:LV63QRGS

load-bearing objection Solid systems paper: real multi-view co-training recipe and external wins; the 36.9→63.2 OOD claim is directionally useful but oversold on N and metric language. the 4 major comments →

arxiv 2607.11643 v1 pith:LV63QRGS submitted 2026-07-13 cs.RO cs.AI

Xiaomi-Robotics-U0: Unified Embodied Synthesis with World Foundation Model

Xinghang Li , Jun Guo , Qiwei Li , Long Qian , Hang Lai , Yueze Wang , Hongyu Yan , Jiahang Cao
show 16 more authors
Xi Chen Jingen Qu Jiaxi Song Nan Sun Hanye Zhao Futeng Liu Wanli Peng Heyun Wang Yunhong Wang Caoyu Xia Jack Zhao Diyun Xiang Hangjun Ye Heng Qu Huaping Liu Jason Li
This is my paper
classification cs.RO cs.AI
keywords embodied world modelsmulti-view scene generationembodied transferautoregressive multimodal modelssynthetic robot datavision-language-action policiesvideo generation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Foundation image and video models generalize well, but they break when robots need multi-view consistency, geometry that matches cameras, and embodiment constraints. Most robot adaptations retrain only on small robot datasets and lose the original visual knowledge. Xiaomi-Robotics-U0 keeps a large pre-trained world model and co-trains it on ordinary image tasks together with multi-view robot scene generation, structured scene transfer, and multi-frame-rate manipulation video. The same model can invent new multi-view robot workspaces from text, edit backgrounds and lighting while freezing robot pose and interaction geometry, and continue those scenes into temporally coherent videos. On human comparisons it beats a strong general image model for multi-view consistency; on WorldArena it ranks first for embodied video; and when its transferred scenes are mixed into policy training, a vision-language-action policy roughly doubles its out-of-distribution progress on real tabletop tasks. The paper therefore claims that foundation world models can act both as embodied world models and as scalable synthetic data engines.

Core claim

Foundation world models can be continually trained under one autoregressive objective on both general image/video tasks and multi-view embodied tasks so that they keep open-domain generation skill while becoming the first unified model that produces high-quality multi-view robot scenes across embodiments, performs structured controllable transfer that preserves geometry and interaction dynamics, generates embodied video, and supplies synthetic data that lifts real robot policy robustness from 36.9% to 63.2% out of distribution.

What carries the argument

Unified next-token prediction over multi-modal sequences that jointly optimizes text-to-image, image editing, multi-view embodied scene generation, structured embodied transfer (workspace / objects / lighting / background disentangled, often conditioned on multi-view depth), and multi-FPS interleaved subtask and video sequences, starting from a large pre-trained autoregressive world foundation model.

Load-bearing premise

Depth maps plus structured text are good enough intermediates for multi-view transfer that keep geometry and robot interactions intact, and that style-only visual transfer (while freezing robot states and actions) is the right way to produce policy-useful data.

What would settle it

Re-run the same real-robot OOD protocol with a matched amount of non-transferred or single-view-only synthetic data: if the policy progress gain disappears, or if multi-view consistency metrics collapse when depth is removed or corrupted, the central data-engine claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. Xiaomi-Robotics-U0 is a 38B multimodal autoregressive model, initialized from EMU3.5/Qwen-3-32B, that jointly trains text-to-image, any-to-image editing, multi-view embodied scene generation, structured embodied transfer (depth + five-factor scene text), and multi-FPS embodied video generation under a single next-token objective. The paper claims this preserves foundation visual knowledge while adding multi-view geometric consistency and robot embodiment constraints, yields SOTA single-step and sequential embodied generation (human wins vs GPT-Image-2.0; first on WorldArena), and that style-transferred multi-view data improves π0.5 OOD robustness on real tabletop tasks from 36.9% to 63.2%.

Significance. If the results hold, the work is a substantial systems contribution to embodied world models: a single large AR model that unifies multi-view scene synthesis, controllable transfer, and video rollout, with public code/checkpoints and a concrete path from foundation generators to synthetic robot data. Strengths include co-training general T2I/X2I with embodied tasks (Table 4 retention), large objective gains on depth/structure/segmentation vs GPT-Image-2 (Table 2), human multi-view preference (Fig. 14), WorldArena EWMScore 73.64 (Table 3), FlashAR+ efficiency, and a real-robot closed-loop data-augmentation study. The data-engine claim is the highest-impact and most fragile part of the package; generation benchmarks alone would still be a solid contribution.

major comments (4)
  1. [Abstract / §3.3] Abstract and §3.3: the headline real-robot result is stated as an “out-of-distribution success rate” of π0.5 from 36.9% to 63.2%, but §3.3 defines and reports only averaged milestone progress Prog(t,g)=ℓ/K over ordered subgoals, not binary success. This is a load-bearing mismatch for the data-engine claim. Please align abstract/intro language with the metric actually used, and report full-success rates alongside progress.
  2. [§3.3] §3.3 Evaluation schedule and Metric: each policy–task–group cell uses only 3 layouts × 3 trials (N=18), with no standard errors, confidence intervals, or significance tests, while Fig. 17 and the abstract treat the ~26-point interference-group gap as decisive evidence that generated data improves OOD robustness. With this N, a few rollouts can dominate the average. Please add per-task variance, more trials or layouts, and a statistical comparison (e.g., bootstrap or paired test) before claiming a robust data-engine effect.
  3. [§2.3.2 / §3.1 / Limitations] §2.3.2, §3.1, Limitations: embodied transfer freezes robot states/action labels and varies only non-task factors via monocular multi-view depth + structured text. Depth estimation artifacts and incomplete disentanglement of task objects are acknowledged but not quantified against policy outcomes. For the claim that transfer “preserves geometric consistency and interaction dynamics” well enough to explain the OOD lift, please report (i) failure cases where depth/structure errors corrupt grasp-relevant geometry and (ii) an ablation of clean vs depth-transferred data quality (or depth-noise injection) on the same π0.5 setup.
  4. [§3.1 / §3.2] §3.1–3.2 baselines: GPT-Image-2 is the main external comparator for multi-view scene generation and transfer. Multi-view reference images are provided to GPT-Image-2 for scene generation, but the protocol for enforcing or scoring cross-view consistency for a single-image model is underspecified relative to U0’s native multi-view training. Please detail prompting, view packing, and any post-processing so the human pairwise wins (Fig. 14) and Table 2 gaps can be interpreted as model capability rather than interface mismatch.
minor comments (6)
  1. [Figures 1–2, 8–13] Fig. 1–2 and several later figures contain garbled/OCR-corrupted text in the provided manuscript PDF; regenerate captions and in-figure labels for readability.
  2. [§2.2] §2.2 claims “38-billion-parameter” while initialization is Qwen-3-32B + IBQ; clarify parameter accounting (tokenizer, heads, FlashAR+ heads) so scale claims are reproducible.
  3. [§3.4 / Table 3] Table 3 WorldArena: report submission date, anonymity codename mapping (UNIS), and whether action-mask conditioning was available to all compared systems, to avoid leaderboard apples-to-oranges concerns.
  4. [§2.4] §2.4 sequential training lists multi-FPS (1/3/5) and several datasets but no ablation of FPS mixture or interleaved subtask-subgoal vs pure video; a short ablation would strengthen the sequential-modeling claim.
  5. [§3.3] Notation: π0.5 / pi_0.5 / pi05_base appear inconsistently; standardize to one form and cite the checkpoint version used for post-training.
  6. [§4] Related work is thorough; a short explicit comparison table (tasks supported: multi-view scene, structured transfer, multi-FPS video, co-trained T2I) vs Dreamer/WAM/Qwen-RobotWorld/Cosmos would help readers place the “first unified” claim.

Circularity Check

0 steps flagged

No significant circularity: empirical systems paper whose claims are tested against external models, leaderboards, and real-robot rollouts not used to define the objective.

full rationale

Xiaomi-Robotics-U0 is a continual-training / multi-task autoregressive systems paper. The load-bearing claims (multi-view scene generation and transfer quality vs GPT-Image-2.0, WorldArena ranking, GenEval/ImgEdit retention, and π0.5 OOD progress lift from style-transferred data) are evaluated on external or held-out protocols: human pairwise preference against GPT-Image-2, the public WorldArena leaderboard, standard GenEval/ImgEdit suites, and real-robot milestone progress under base vs interference layouts. Depth-plus-structured-text conditioning and freezing of robot states/action labels are design choices for the transfer pipeline, not tautological definitions that force the reported metrics. In-house MiBot data is ordinary training corpus, not a self-citation that justifies the central results. There is no uniqueness theorem, fitted parameter renamed as prediction, or derivation that reduces by construction to its inputs. Metric wording mismatch (abstract “success rate” vs body milestone progress) and small-N real-robot evaluation are correctness/robustness concerns, not circularity. Score 0 is therefore appropriate.

Axiom & Free-Parameter Ledger

5 free parameters · 6 axioms · 3 invented entities

Load-bearing content is mostly engineering assumptions and training design choices, not free physical constants. The central story depends on discrete AR image modeling, depth as geometry proxy, multi-task co-training to avoid forgetting, and style-only augmentation for policy robustness. Free parameters are loss weights, temporal FPS set, data mix, and experiment scale choices.

free parameters (5)
  • FlashAR+ directional/distill loss weights = 0.05 / 0.05 / 0.2
    L = L_fuse + 0.05 L_h + 0.05 L_v + 0.2 L_distill; weights chosen to stabilize H/V fusion and prevent gate collapse (§2.5.1).
  • Multi-FPS video training rates = 1, 3, 5
    Embodied videos trained at FPS ∈ {1,3,5} to span long-horizon vs fine dynamics (§2.4); discrete design choice affecting sequential capability.
  • Per-task clean vs style-augmented data volume = ~40h + ~40h per task
    ≈40 h real demos + ≈40 h Xiaomi-Robotics-U0 style-transferred episodes per task for π0.5 SFT (§3.3); mixture ratio is a free experimental knob for the OOD claim.
  • Domain/task sample reweighting
    Hierarchical reweighting prioritizes long-tail skills and downsamples redundant embodiment-task-object combos (§2.3.2); exact weights not fully specified but control the training distribution.
  • Model scale / initialization = 38B / EMU3.5
    38B parameters initialized from EMU3.5 (Qwen-3-32B + IBQ); scale and base checkpoint are design choices the results depend on.
axioms (6)
  • domain assumption Unified next-token prediction over discrete IBQ image tokens plus text/control tokens can learn multi-view geometric consistency and robot embodiment constraints.
    Core modeling choice in §2.1–2.2; no separate 3D or multi-view consistency loss is required by construction.
  • domain assumption Monocular/video depth maps plus structured text suffice as geometric conditioning for photorealistic multi-view transfer that preserves interaction dynamics.
    Embodied transfer formulation and labeling pipeline (§2.3.2, §3.1); authors later note depth artifacts as a limitation.
  • domain assumption Joint continual training on general T2I/X2I with embodied tasks preserves foundation visual knowledge while adding robot-centric skills.
    Stated motivation and single-step training setup (§1, §2.4); GenEval/ImgEdit used as partial check.
  • domain assumption Varying only non-task visual factors (workspace, background, lighting, distractors) while freezing robot states and action labels improves OOD policy robustness.
    Zero-shot augmentation protocol and real-world experiment design (§3.1, §3.3).
  • domain assumption Human pairwise preference is a valid primary measure of multi-view geometric consistency when automated 3D metrics are unavailable.
    Scene generation evaluation protocol (§3.2).
  • standard math Standard transformer AR training and classifier-free guidance behave as in prior multimodal generators.
    Background ML practice assumed throughout §2.
invented entities (3)
  • Structured five-factor embodied scene control (workspace, task objects, irrelevant objects, lighting, background) independent evidence
    purpose: Disentangle editable scene dimensions for controllable multi-view transfer and scalable augmentation while holding robot geometry fixed.
    Introduced as a labeling and control formulation (§2.3.2, contributions); factors are annotation conventions, not new physics.
  • FlashAR+ X2I extension (prefix-conditioned step-causal anti-diagonal decoding with H/V gated fusion) independent evidence
    purpose: Accelerate multi-reference image generation without breaking multi-source conditioning.
    Post-training/inference adaptation of FlashAR for this model’s X2I setting (§2.5); engineering construct validated by latency/quality tables.
  • Xiaomi-Robotics-U0 unified task suite (scene gen + transfer + multi-FPS video + T2I/X2I under one AR objective) no independent evidence
    purpose: Single model spanning foundation and embodied synthesis as a data engine.
    The paper’s primary system artifact; evidence is empirical benchmarks and robot experiments, not external prior existence of this exact suite.

pith-pipeline@v1.1.0-grok45 · 44789 in / 4309 out tokens · 45481 ms · 2026-07-14T04:09:23.541966+00:00 · methodology

0 comments
read the original abstract

Recent foundation image and video generation models offer strong generalization and controllability, but their direct application to embodied scenarios is limited by requirements for multi-view consistency, geometric coherence, and robot embodiment constraints. Existing methods typically adapt foundation models with limited robot data, often sacrificing visual knowledge acquired during large-scale pre-training. We present Xiaomi-Robotics-U0, a 38-billion-parameter multimodal autoregressive model for unified embodied synthesis. It treats embodied generation as an extension of foundation image and video generation and jointly optimizes text-to-image generation, image editing, embodied scene generation, embodied transfer, and embodied video generation. This unified framework preserves the generalization of the pre-trained world foundation model while adapting it to embodied settings. Xiaomi-Robotics-U0 is the first model to support high-quality multi-view scene generation across multiple robot embodiments and to introduce structured, controllable embodied transfer for fine-grained editing while preserving multi-view consistency and interaction dynamics. It achieves state-of-the-art results on single-step and sequential generation tasks, outperforming GPT-Image-2.0 in human evaluations of embodied scene generation and transfer, ranking first on World Arena for embodied video generation, and improving the out-of-distribution success rate of pi_0.5 from 36.9% to 63.2% on challenging real-world manipulation tasks. These results show that foundation world models can serve both as embodied world models and scalable data engines for embodied intelligence. Code and checkpoints are available at https://robotics.xiaomi.com/xiaomi-robotics-u0.html.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

73 extracted references · 35 linked inside Pith

  1. [1]

    Cosmos 3: Omnimodal world models for physical ai.arXiv preprint arXiv:2606.02800, 2026

    Niket Agarwal, Arslan Ali, Jon Allen, Martin Antolini, Adeline Aubame, Alisson Azzolini, Junjie Bai, Maciej Bala, Yogesh Balaji, Josh Bapst, et al. Cosmos 3: Omnimodal world models for physical ai.arXiv preprint arXiv:2606.02800, 2026

    Pith/arXiv arXiv 2026

  2. [2]

    Egowalk: A multimodal dataset for robot navigation in the wild.arXiv preprint arXiv:2505.21282, 2025

    Timur Akhtyamov, Mohamad Al Mdfaa, Javier Antonio Ramirez Benavides, Arthur Nigmatzyanov, Sergey Bakulin, German Devchich, Denis Fatykhov, Diego Ruiz Salinas, Alexander Mazurov, Kristina Zipa, et al. Egowalk: A multimodal dataset for robot navigation in the wild.arXiv preprint arXiv:2505.21282, 2025

    Pith/arXiv arXiv 2025

  3. [3]

    Qwen3-VL technical report.arXiv preprint arXiv:2511.21631, 2025

    Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, et al. Qwen3-VL technical report.arXiv preprint arXiv:2511.21631, 2025

    Pith/arXiv arXiv 2025

  4. [4]

    Stable video diffusion: Scaling latent video diffusion models to large datasets.arXiv preprint arXiv:2311.15127, 2023

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets.arXiv preprint arXiv:2311.15127, 2023

    Pith/arXiv arXiv 2023

  5. [5]

    RT-1: Robotics transformer for real-world control at scale.arXiv preprint arXiv:2212.06817, 2022

    Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Jasmine Hsu, et al. RT-1: Robotics transformer for real-world control at scale.arXiv preprint arXiv:2212.06817, 2022

    Pith/arXiv arXiv 2022

  6. [6]

    Instructpix2pix: Learning to follow image editing instructions

    Tim Brooks, Aleksander Holynski, and Alexei A Efros. Instructpix2pix: Learning to follow image editing instructions. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 18392–18402, 2023

    2023

  7. [7]

    Genie: Generative interactive environments

    Jake Bruce, Michael D Dennis, Ashley Edwards, Jack Parker-Holder, Yuge Shi, Edward Hughes, Matthew Lai, Aditi Mavalankar, Richie Steigerwald, Chris Apps, et al. Genie: Generative interactive environments. InForty-first International Conference on Machine Learning, 2024

    2024

  8. [8]

    AgiBot world colosseo: A large-scale manipulation platform for scalable and intelligent embodied systems

    Qingwen Bu, Jisong Cai, Li Chen, Xiuqi Cui, Yan Ding, Siyuan Feng, Shenyuan Gao, Xindong He, Xuan Hu, Xu Huang, et al. AgiBot world colosseo: A large-scale manipulation platform for scalable and intelligent embodied systems. arXiv preprint arXiv:2503.06669, 2025

    Pith/arXiv arXiv 2025

  9. [9]

    Sharegpt4v: Improving large multi-modal models with better captions

    Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Conghui He, Jiaqi Wang, Feng Zhao, and Dahua Lin. Sharegpt4v: Improving large multi-modal models with better captions. InEuropean Conference on Computer Vision, pages 370–387. Springer, 2024

    2024

  10. [10]

    Video depth anything: Consistent depth estimation for super-long videos

    Sili Chen, Hengkai Guo, Shengnan Zhu, Feihu Zhang, Zilong Huang, Jiashi Feng, and Bingyi Kang. Video depth anything: Consistent depth estimation for super-long videos. InProceedings of the Computer Vision and Pattern Recognition Conference, pages 22831–22840, 2025

    2025

  11. [11]

    Robotwin 2.0: A scalable data generator and benchmark with strong domain randomization for robust bimanual robotic manipulation.arXiv preprint arXiv:2506.18088, 2025

    Tianxing Chen, Zanxin Chen, Baijun Chen, Zijian Cai, Yibin Liu, Zixuan Li, Qiwei Liang, Xianliang Lin, Yiheng Ge, Zhenyu Gu, et al. Robotwin 2.0: A scalable data generator and benchmark with strong domain randomization for robust bimanual robotic manipulation.arXiv preprint arXiv:2506.18088, 2025

    Pith/arXiv arXiv 2025

  12. [12]

    Anydoor: Zero-shot object-level image customization

    Xi Chen, Lianghua Huang, Yu Liu, Yujun Shen, Deli Zhao, and Hengshuang Zhao. Anydoor: Zero-shot object-level image customization. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6593–6602, 2024

    2024

  13. [13]

    Janus-pro: Unified multimodal understanding and generation with data and model scaling

    Xiaokang Chen, Zhiyu Wu, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, and Chong Ruan. Janus-pro: Unified multimodal understanding and generation with data and model scaling. arXiv preprint arXiv:2501.17811, 2025

    Pith/arXiv arXiv 2025

  14. [14]

    Abot-physworld: Interactive world foundation model for robotic manipulation with physics alignment.arXiv preprint arXiv:2603.23376, 2026

    Yuzhi Chen, Ronghan Chen, Dongjie Huo, Yandan Yang, Dekang Qi, Haoyun Liu, Tong Lin, Shuang Zeng, Junjin Xiao, Xinyuan Chang, et al. Abot-physworld: Interactive world foundation model for robotic manipulation with physics alignment.arXiv preprint arXiv:2603.23376, 2026

    arXiv 2026

  15. [15]

    Open X-Embodiment: Robotic learning datasets and RT-X models.https://arxiv.org/abs/2310.08864, 2023

    Open X-Embodiment Collaboration, Abby O’Neill, Abdul Rehman, Abhinav Gupta, Abhiram Maddukuri, Abhishek Gupta, Abhishek Padalkar, Abraham Lee, Acorn Pooley, Agrim Gupta, Ajay Mandlekar, Ajinkya Jain, Albert Tung, Alex Bewley, Alex Herzog, Alex Irpan, et al. Open X-Embodiment: Robotic learning datasets and RT-X models.https://arxiv.org/abs/2310.08864, 2023

    Pith/arXiv arXiv 2023

  16. [16]

    Yufeng Cui, Honghao Chen, Haoge Deng, Xu Huang, Xinghang Li, Jirong Liu, Yang Liu, Zhuoyan Luo, Jin- sheng Wang, Wenxuan Wang, et al. Emu3. 5: Native multimodal models are world learners.arXiv preprint arXiv:2510.26583, 2025. 35

    Pith/arXiv arXiv 2025

  17. [17]

    Emerging properties in unified multimodal pretraining.arXiv preprint arXiv:2505.14683, 2025

    Chaorui Deng, Deyao Zhu, Kunchang Li, Chenhui Gou, Feng Li, Zeyu Wang, Shu Zhong, Weihao Yu, Xiaonan Nie, Ziang Song, et al. Emerging properties in unified multimodal pretraining.arXiv preprint arXiv:2505.14683, 2025

    Pith/arXiv arXiv 2025

  18. [18]

    Datacomp: In search of the next generation of multimodal datasets

    Samir Yitzhak Gadre, Gabriel Ilharco, Alex Fang, Jonathan Hayase, Georgios Smyrnis, Thao Nguyen, Ryan Marten, Mitchell Wortsman, Dhruba Ghosh, Jieyu Zhang, et al. Datacomp: In search of the next generation of multimodal datasets. Advancesin Neural Information Processing Systems, 36:27092–27112, 2023

    2023

  19. [19]

    Cat3d: Create anything in 3d with multi-view diffusion models.arXiv preprint arXiv:2405.10314, 2024

    Ruiqi Gao, Aleksander Holynski, Philipp Henzler, Arthur Brussee, Ricardo Martin-Brualla, Pratul Srinivasan, Jonathan T Barron, and Ben Poole. Cat3d: Create anything in 3d with multi-view diffusion models.arXiv preprint arXiv:2405.10314, 2024

    Pith/arXiv arXiv 2024

  20. [20]

    Geneval: An object-focused framework for evaluating text-to-image alignment.Advancesin Neural Information Processing Systems, 36:52132–52152, 2023

    Dhruba Ghosh, Hannaneh Hajishirzi, and Ludwig Schmidt. Geneval: An object-focused framework for evaluating text-to-image alignment.Advancesin Neural Information Processing Systems, 36:52132–52152, 2023

    2023

  21. [21]

    Ego4d: Around the world in 3,000 hours of egocentric video

    Kristen Grauman, Andrew Westbury, Eugene Byrne, Zachary Chavis, Antonino Furnari, Rohit Girdhar, Jackson Hamburger, Hao Jiang, Miao Liu, Xingyu Liu, et al. Ego4d: Around the world in 3,000 hours of egocentric video. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 18995–19012, 2022

    2022

  22. [22]

    Dream to control: Learning behaviors by latent imagination.arXiv preprint arXiv:1912.01603, 2019

    Danijar Hafner, Timothy Lillicrap, Jimmy Ba, and Mohammad Norouzi. Dream to control: Learning behaviors by latent imagination.arXiv preprint arXiv:1912.01603, 2019

    Pith/arXiv arXiv 1912

  23. [23]

    arXiv preprint arXiv:2504.16054, 2025

    Physical Intelligence, Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, et al.π0.5: A Vision-Language-Action Model with Open-World Generalization. arXiv preprint arXiv:2504.16054, 2025

    Pith/arXiv arXiv 2025

  24. [24]

    Galaxea open-world dataset and g0 dual-system vla model.arXiv preprint arXiv:2509.00576, 2025

    Tao Jiang, Tianyuan Yuan, Yicheng Liu, Chenhao Lu, Jianning Cui, Xiao Liu, Shuiqi Cheng, Jiyang Gao, Huazhe Xu, and Hang Zhao. Galaxea open-world dataset and g0 dual-system vla model.arXiv preprint arXiv:2509.00576, 2025

    Pith/arXiv arXiv 2025

  25. [25]

    Droid: A large-scale in-the-wild robot manipulation dataset.arXiv preprint arXiv:2403.12945, 2024

    Alexander Khazatsky, Karl Pertsch, Suraj Nair, Ashwin Balakrishna, Sudeep Dasari, Siddharth Karamcheti, Soroush Nasiriany, Mohan Kumar Srirama, Lawrence Yunliang Chen, Kirsty Ellis, et al. Droid: A large-scale in-the-wild robot manipulation dataset.arXiv preprint arXiv:2403.12945, 2024

    Pith/arXiv arXiv 2024

  26. [26]

    Gonzalez, Hao Zhang, and Ion Stoica

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, 2023

    2023

  27. [27]

    Flux.1 kontext: Flow matching for in-context image generation and editing in latent space, 2025

    Black Forest Labs, Stephen Batifol, Andreas Blattmann, Frederic Boesel, Saksham Consul, Cyril Diagne, Tim Dockhorn, Jack English, Zion English, Patrick Esser, Sumith Kulal, Kyle Lacey, Yam Levi, Cheng Li, Dominik Lorenz, Jonas Müller, Dustin Podell, Robin Rombach, Harry Saini, Axel Sauer, and Luke Smith. Flux.1 kontext: Flow matching for in-context image ...

    Pith/arXiv arXiv 2025

  28. [28]

    Causal world modeling for robot control.arXiv preprint arXiv:2601.21998, 2026

    Lin Li, Qihang Zhang, Yiming Luo, Shuai Yang, Ruilin Wang, Fei Han, Mingrui Yu, Zelin Gao, Nan Xue, Xing Zhu, et al. Causal world modeling for robot control.arXiv preprint arXiv:2601.21998, 2026

    Pith/arXiv arXiv 2026

  29. [29]

    Era3d: High-resolution multiview diffusion using efficient row-wise attention.Advancesin Neural Information Processing Systems, 37:55975–56000, 2024

    Peng Li, Yuan Liu, Xiaoxiao Long, Feihu Zhang, Cheng Lin, Mengfei Li, Xingqun Qi, Shanghang Zhang, Wenhan Luo, Ping Tan, et al. Era3d: High-resolution multiview diffusion using efficient row-wise attention.Advancesin Neural Information Processing Systems, 37:55975–56000, 2024

    2024

  30. [30]

    A comprehensive survey on world models for embodied ai

    Xinqing Li, Xin He, Le Zhang, Min Wu, Xiaoli Li, and Yun Liu. A comprehensive survey on world models for embodied ai. arXiv preprint arXiv:2510.16732, 2025

    Pith/arXiv arXiv 2025

  31. [31]

    Dl3dv-10k: A large-scale scene dataset for deep learning-based 3d vision

    Lu Ling, Yichen Sheng, Zhi Tu, Wentian Zhao, Cheng Xin, Kun Wan, Lantao Yu, Qianyu Guo, Zixun Yu, Yawen Lu, et al. Dl3dv-10k: A large-scale scene dataset for deep learning-based 3d vision. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22160–22169, 2024

    2024

  32. [32]

    Aligning cyber space with physical world: A comprehensive survey on embodied ai.IEEE/ASME Transactions on Mechatronics, 2025

    Yang Liu, Weixing Chen, Yongjie Bai, Xiaodan Liang, Guanbin Li, Wen Gao, and Liang Lin. Aligning cyber space with physical world: A comprehensive survey on embodied ai.IEEE/ASME Transactions on Mechatronics, 2025

    2025

  33. [33]

    Syncdreamer: Generating multiview-consistent images from a single-view image

    Yuan Liu, Cheng Lin, Zijiao Zeng, Xiaoxiao Long, Lingjie Liu, Taku Komura, and Wenping Wang. Syncdreamer: Generating multiview-consistent images from a single-view image. In International conference on learning representations, volume 2024, pages 27676–27697, 2024. 36

    2024

  34. [34]

    Scaling world model for hierarchical manipulation policies.arXiv preprintarXiv:2602.10983, 2026

    Qian Long, Yueze Wang, Jiaxi Song, Junbo Zhang, Peiyan Li, Wenxuan Wang, Yuqi Wang, Haoyang Li, Shaoxuan Xie, Guocai Yao, et al. Scaling world model for hierarchical manipulation policies.arXiv preprintarXiv:2602.10983, 2026

    arXiv 2026

  35. [35]

    Wonder3d: Single image to 3d using cross-domain diffusion

    Xiaoxiao Long, Yuan-Chen Guo, Cheng Lin, Yuan Liu, Zhiyang Dou, Lingjie Liu, Yuexin Ma, Song-Hai Zhang, Marc Habermann, Christian Theobalt, et al. Wonder3d: Single image to 3d using cross-domain diffusion. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9970–9980, 2024

    2024

  36. [36]

    A survey: Learning embodied intelligence from physical simulators and world models

    Xiaoxiao Long, Qingrui Zhao, Kaiwen Zhang, Zihao Zhang, Dingrui Wang, Yumeng Liu, Zhengjie Shu, Yi Lu, Shouzheng Wang, Xinzhe Wei, et al. A survey: Learning embodied intelligence from physical simulators and world models. arXiv preprint arXiv:2507.00917, 2025

    Pith/arXiv arXiv 2025

  37. [37]

    Leland McInnes, John Healy, and S. Astels. hdbscan: Hierarchical density based clustering.J. Open Source Softw., 2:205, 2017. URLhttps://api.semanticscholar.org/CorpusID:53231359

    2017

  38. [38]

    Coyo-700m: Image-text pair dataset.https://github.com/kakaobrain/coyo-dataset, 2022

    Byeon Minwoo, Park Beomhee, Kim Haecheon, Lee Sungjun, Woonhyuk Baek, and Kim Saehoon. Coyo-700m: Image-text pair dataset.https://github.com/kakaobrain/coyo-dataset, 2022

    2022

  39. [39]

    Maniskill: Generalizable manipulation skill benchmark with large-scale demonstrations.arXiv preprint arXiv:2107.14483, 2021

    Tongzhou Mu, Zhan Ling, Fanbo Xiang, Derek Yang, Xuanlin Li, Stone Tao, Zhiao Huang, Zhiwei Jia, and Hao Su. Maniskill: Generalizable manipulation skill benchmark with large-scale demonstrations.arXiv preprint arXiv:2107.14483, 2021

    Pith/arXiv arXiv 2021

  40. [40]

    Robocasa: Large-scale simulation of everyday tasks for generalist robots

    Soroush Nasiriany, Abhiram Maddukuri, Lance Zhang, Adeet Parikh, Aaron Lo, Abhishek Joshi, Ajay Mandlekar, and Yuke Zhu. Robocasa: Large-scale simulation of everyday tasks for generalist robots. arXiv preprint arXiv:2406.02523, 2024

    Pith/arXiv arXiv 2024

  41. [41]

    Video generation models as world simulators

    OpenAI. Video generation models as world simulators. https://openai.com/research/ video-generation-models-as-world-simulators, 2024. Technical report

    2024

  42. [42]

    Gpt image api

    OpenAI. Gpt image api. https://platform.openai.com/docs/guides/image-generation, 2025. Accessed: 2026-07-03

    2025

  43. [43]

    Open x-embodiment: Robotic learning datasets and rt-x models: Open x-embodiment collaboration 0

    Abby O’Neill, Abdul Rehman, Abhiram Maddukuri, Abhishek Gupta, Abhishek Padalkar, Abraham Lee, Acorn Pooley, Agrim Gupta, Ajay Mandlekar, Ajinkya Jain, et al. Open x-embodiment: Robotic learning datasets and rt-x models: Open x-embodiment collaboration 0. In2024 IEEE International Conference on Robotics and Automation (ICRA), pages 6892–6903. IEEE, 2024

    2024

  44. [44]

    Genie 2: A large-scale foundation world model

    Jack Parker-Holder, Philip Ball, Jake Bruce, Vibhavari Dasagi, Kristian Holsheimer, Christos Kaplanis, Alexandre Moufarek, Guy Scully, Jeremy Shar, Jimmy Shi, Stephen Spencer, Jessica Yung, Michael Dennis, Sultan Kenjeyev, Shangbang Long, Vlad Mnih, Harris Chan, Maxime Gazeau, Bonnie Li, Fabio Pardo, Luyu Wang, Lei Zhang, Frederic Besse, Tim Harley, Anna ...

    2024

  45. [45]

    Sdxl: Improving latent diffusion models for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. InInternational Conference on Learning Representations, volume 2024, pages 1862–1874, 2024

    2024

  46. [46]

    Movie gen: A cast of media foundation models.arXiv preprint arXiv:2410.13720, 2024

    Adam Polyak, Amit Zohar, Andrew Brown, Andros Tjandra, Animesh Sinha, Ann Lee, Apoorv Vyas, Bowen Shi, Chih-Yao Ma, Ching-Yao Chuang, et al. Movie gen: A cast of media foundation models.arXiv preprint arXiv:2410.13720, 2024

    Pith/arXiv arXiv 2024

  47. [47]

    Cosmos-drive-dreams: Scalable synthetic driving data generation with world foundation models.arXiv preprint arXiv:2506.09042, 2025

    Xuanchi Ren, Yifan Lu, Tianshi Cao, Ruiyuan Gao, Shengyu Huang, Amirmojtaba Sabour, Tianchang Shen, Tobias Pfaff, Jay Zhangjie Wu, Runjian Chen, et al. Cosmos-drive-dreams: Scalable synthetic driving data generation with world foundation models.arXiv preprint arXiv:2506.09042, 2025

    Pith/arXiv arXiv 2025

  48. [48]

    Laion-5b: An open large-scale dataset for training next generation image-text models.Advances in neural information processing systems, 35:25278–25294, 2022

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. Laion-5b: An open large-scale dataset for training next generation image-text models.Advances in neural information processing systems, 35:25278–25294, 2022

    2022

  49. [49]

    Worldarena: A unified benchmark for evaluating perception and functional utility of embodied world models

    Yu Shang, Zhuohang Li, Yiding Ma, Weikang Su, Xin Jin, Ziyou Wang, Lei Jin, Xin Zhang, Yinzhou Tang, Haisheng Su, et al. Worldarena: A unified benchmark for evaluating perception and functional utility of embodied world models. arXiv preprint arXiv:2602.08971, 2026. 37

    arXiv 2026

  50. [50]

    Roboscape: Physics-informed embodied world model.Advancesin Neural Information Processing Systems, 38:63674–63698, 2026

    Yu Shang, Xin Zhang, Yinzhou Tang, Lei Jin, Chen Gao, Wei Wu, and Yong Li. Roboscape: Physics-informed embodied world model.Advancesin Neural Information Processing Systems, 38:63674–63698, 2026

    2026

  51. [51]

    Scalable image tokenization with index backpropagation quantization

    Fengyuan Shi, Zhuoyan Luo, Yixiao Ge, Yujiu Yang, Ying Shan, and Limin Wang. Scalable image tokenization with index backpropagation quantization. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 16037–16046, 2025

    2025

  52. [52]

    Zero123++: a single image to consistent multi-view diffusion base model.arXiv preprint arXiv:2310.15110, 2023

    Ruoxi Shi, Hansheng Chen, Zhuoyang Zhang, Minghua Liu, Chao Xu, Xinyue Wei, Linghao Chen, Chong Zeng, and Hao Su. Zero123++: a single image to consistent multi-view diffusion base model.arXiv preprint arXiv:2310.15110, 2023

    Pith/arXiv arXiv 2023

  53. [53]

    Mvdream: Multi-view diffusion for 3d generation

    Yichun Shi, Peng Wang, Jianglong Ye, Long Mai, Kejie Li, and Xiao Yang. Mvdream: Multi-view diffusion for 3d generation. In International conference on learning representations, volume 2024, pages 39838–39859, 2024

    2024

  54. [54]

    Mvdiffusion++: A dense high-resolution multi-view diffusion model for single or sparse-view 3d object reconstruction

    Shitao Tang, Jiacheng Chen, Dilin Wang, Chengzhou Tang, Fuyang Zhang, Yuchen Fan, Vikas Chandra, Yasutaka Furukawa, and Rakesh Ranjan. Mvdiffusion++: A dense high-resolution multi-view diffusion model for single or sparse-view 3d object reconstruction. InEuropean Conference on Computer Vision, pages 175–191. Springer, 2024

    2024

  55. [55]

    Chameleon: Mixed-modal early-fusion foundation models, 2024

    Chameleon Team. Chameleon: Mixed-modal early-fusion foundation models, 2024. URL https://arxiv. org/abs/2405.09818, 9(8), 2024

    Pith/arXiv arXiv 2024

  56. [56]

    Motubrain: An advanced world action model for robot control.arXiv preprint arXiv:2604.27792, 2026

    MotuBrain Team, Chendong Xiang, Fan Bao, Haitian Liu, Hengkai Tan, Hongzhe Bi, James Li, Jiabao Liu, Jingrui Pang, Kiro Jing, et al. Motubrain: An advanced world action model for robot control.arXiv preprint arXiv:2604.27792, 2026

    Pith/arXiv arXiv 2026

  57. [57]

    Interndata-a1: Pioneering high-fidelity synthetic data for pre-training generalist policy

    Yang Tian, Yuyin Yang, Yiman Xie, Zetao Cai, Xu Shi, Ning Gao, Hangxu Liu, Xuekun Jiang, Zherui Qiu, Feng Yuan, et al. Interndata-a1: Pioneering high-fidelity synthetic data for pre-training generalist policy. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 976–985, 2026

    2026

  58. [58]

    Bridgedata v2: A dataset for robot learning at scale

    Homer Rich Walke, Kevin Black, Tony Z Zhao, Quan Vuong, Chongyi Zheng, Philippe Hansen-Estruch, An- dre Wang He, Vivek Myers, Moo Jin Kim, Max Du, et al. Bridgedata v2: A dataset for robot learning at scale. In Conference on Robot Learning, pages 1723–1736. PMLR, 2023

    2023

  59. [59]

    Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314, 2025

    Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianx- iao Yang, et al. Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314, 2025

    Pith/arXiv arXiv 2025

  60. [60]

    Janus: Decoupling visual encoding for unified multimodal understanding and generation

    Chengyue Wu, Xiaokang Chen, Zhiyu Wu, Yiyang Ma, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, Chong Ruan, et al. Janus: Decoupling visual encoding for unified multimodal understanding and generation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 12966–12977, 2025

    2025

  61. [61]

    Robomind: Benchmark on multi-embodiment intelligence normative data for robot manipulation

    Kun Wu, Chengkai Hou, Jiaming Liu, Zhengping Che, Xiaozhu Ju, Zhuqin Yang, Meng Li, Yinuo Zhao, Zhiyuan Xu, Guang Yang, et al. Robomind: Benchmark on multi-embodiment intelligence normative data for robot manipulation. arXiv preprint arXiv:2412.13877, 2024

    Pith/arXiv arXiv 2024

  62. [62]

    Robocoin: An open-sourced bimanual robotic data collection for integrated manipulation

    Shihan Wu, Xuecheng Liu, Shaoxuan Xie, Pengwei Wang, Xinghang Li, Bowen Yang, Zhe Li, Kai Zhu, Hongyu Wu, Yiheng Liu, et al. Robocoin: An open-sourced bimanual robotic data collection for integrated manipulation. arXiv preprint arXiv:2511.17441, 2025

    Pith/arXiv arXiv 2025

  63. [63]

    Omnigen: Unified image generation

    Shitao Xiao, Yueze Wang, Junjie Zhou, Huaying Yuan, Xingrun Xing, Ruiran Yan, Chaofan Li, Shuting Wang, Tiejun Huang, and Zheng Liu. Omnigen: Unified image generation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13294–13304, 2025

    2025

  64. [64]

    Show-o: One single transformer to unify multimodal understanding and generation

    Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang, Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhenheng Yang, and Mike Zheng Shou. Show-o: One single transformer to unify multimodal understanding and generation. InInternational Conference on Learning Representations, volume 2025, pages 28240–28264, 2025

    2025

  65. [65]

    Gigaworld-policy: An efficient action-centered world–action model.arXiv preprint arXiv:2603.17240, 2026

    Angen Ye, Boyuan Wang, Chaojun Ni, Guan Huang, Guosheng Zhao, Hao Li, Hengtao Li, Jie Li, Jindi Lv, Jingyu Liu, et al. Gigaworld-policy: An efficient action-centered world–action model.arXiv preprint arXiv:2603.17240, 2026

    arXiv 2026

  66. [66]

    World action models are zero-shot policies.arXiv preprint arXiv:2602.15922, 2026

    Seonghyeon Ye, Yunhao Ge, Kaiyuan Zheng, Shenyuan Gao, Sihyun Yu, George Kurian, Suneel Indupuru, You Liang Tan, Chuning Zhu, Jiannan Xiang, et al. World action models are zero-shot policies.arXiv preprint arXiv:2602.15922, 2026. 38

    Pith/arXiv arXiv 2026

  67. [67]

    Imgedit: A unified image editing dataset and benchmark.Advancesin Neural Information Processing Systems, 38, 2026

    Yang Ye, Xianyi He, Zongjian Li, Shenghai Yuan, Zhiyuan Yan, Bohan Hou, Li Yuan, et al. Imgedit: A unified image editing dataset and benchmark.Advancesin Neural Information Processing Systems, 38, 2026

    2026

  68. [68]

    Scannet++: A high-fidelity dataset of 3d indoor scenes

    Chandan Yeshwanth, Yueh-Cheng Liu, Matthias Nießner, and Angela Dai. Scannet++: A high-fidelity dataset of 3d indoor scenes. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 12–22, 2023

    2023

  69. [69]

    Genie sim 3.0 : A high-fidelity comprehensive simulation platform for humanoid robot, 2026

    Chenghao Yin, Da Huang, Di Yang, Jichao Wang, Nanshu Zhao, Chen Xu, Wenjun Sun, Linjie Hou, Zhijun Li, Junhui Wu, Zhaobo Liu, Zhen Xiao, Sheng Zhang, Lei Bao, Rui Feng, Zhenquan Pang, Jiayu Li, Qian Wang, and Maoqing Yao. Genie sim 3.0 : A high-fidelity comprehensive simulation platform for humanoid robot, 2026. URL https://arxiv.org/abs/2601.02078

    Pith/arXiv arXiv 2026

  70. [70]

    Fast-wam: Do world action models need test-time future imagination? arXiv preprint arXiv:2603.16666, 2026

    Tianyuan Yuan, Zibin Dong, Yicheng Liu, and Hang Zhao. Fast-wam: Do world action models need test-time future imagination? arXiv preprint arXiv:2603.16666, 2026

    Pith/arXiv arXiv 2026

  71. [71]

    Scaling behavior cloning improves causal reasoning: An open model for real-time video game playing.arXiv preprint arXiv:2601.04575, 2026

    Yuguang Yue, Irakli Salia, Samuel Hunt, Chris Green, Wenzhe Shi, and Jonathan J Hunt. Scaling behavior cloning improves causal reasoning: An open model for real-time video game playing.arXiv preprint arXiv:2601.04575, 2026

    arXiv 2026

  72. [72]

    Qwen-robotworld technical report: Unifying embodied world modeling through language- conditioned video generation.arXiv preprint arXiv:2606.17030, 2026

    Jie Zhang, Xiaoyue Chen, Anzhe Chen, Chenxu Lv, Deqing Li, Gengze Zhou, Hang Yin, Haoqi Yuan, Haoyang Li, Jiahao Li, et al. Qwen-robotworld technical report: Unifying embodied world modeling through language- conditioned video generation.arXiv preprint arXiv:2606.17030, 2026

    Pith/arXiv arXiv 2026

  73. [73]

    Flashar: Efficient post-training acceleration for autoregressive image generation, 2026

    Junkang Zhou, Yefei He, Feng Chen, Weijie Wang, and Bohan Zhuang. Flashar: Efficient post-training acceleration for autoregressive image generation, 2026. URLhttps://arxiv.org/abs/2605.09430. 39

    Pith/arXiv arXiv 2026