The reviewed record of science sign in
Pith

arxiv: 2607.06216 · v1 · pith:VOFYHDGY · submitted 2026-07-07 · cs.CV

MoWorld: A Flash World Model

Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel 2026-07-08 12:55 UTCglm-5.2pith:VOFYHDGYrecord.jsonopen to challenge →

classification cs.CV
keywords world modelreal-time inferenceNPUdiffusion distillationautoregressive generationvideo generationcamera controlmixture of experts
0
0 comments X

The pith

Real-time world model runs at 50 FPS on NPUs, no GPU required

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

MoWorld argues that the bottleneck for world models is no longer model capacity but deployment practicality. The paper presents an end-to-end pipeline — spanning 3D-native data generation, curriculum pretraining, autoregressive distillation, and NPU system optimization — that collectively enables real-time interactive world simulation at up to 50 FPS on neural processing units rather than high-end GPUs. The central mechanism is a four-part co-design: a geometry-aware data engine that produces geometrically consistent training samples with accurate camera poses; a curriculum cross-frame training strategy that progressively extends video sequence length from short clips to 2000 frames; a Self-Forcing distillation framework that compresses a bidirectional diffusion teacher into a 4-step autoregressive student while correcting rollout errors via distribution matching; and a mixed-precision parallel inference system with on-demand module loading and fused attention kernels tailored for NPU memory constraints. The paper claims this pipeline reduces average inference cost to 30-50% of existing world models while maintaining competitive visual quality, subject consistency, and camera controllability. If correct, this means interactive world simulation — generating future visual states conditioned on user actions in real time — becomes feasible on widely available edge hardware rather than requiring data-center GPUs, which would lower the barrier for embodied intelligence, autonomous driving, cloud gaming, and digital twin applications.

Core claim

The paper's central claim is that a 14B-parameter mixture-of-experts video diffusion model can be converted into a real-time interactive world model running at up to 50 FPS on NPUs through a specific distillation and system co-design chain. The load-bearing technical mechanism is the Self-Forcing distillation: the student model generates full videos through the same autoregressive chunk-by-chunk rollout used at inference, while a frozen teacher and a fake model provide a distribution-matching gradient (the difference between real and fake velocity fields) that corrects rollout errors. This allows the 50-step diffusion process to be compressed to 4 steps without requiring expensive teacher OD

What carries the argument

Self-Forcing distillation with distribution matching (DMD-style), History Context selection strategy, Plücker camera embeddings, curriculum cross-frame training, NPU mixed-precision inference with hierarchical sequence parallelism

If this is right

  • If the 4-step distilled autoregressive student preserves teacher quality, interactive world models could be deployed on edge devices (phones, vehicles, robots) without GPU infrastructure, enabling closed-loop perception and planning at interactive latencies.
  • The 3D-native data engine approach — where training data includes explicit camera geometry rather than loosely paired video-text clips — could become a standard for world model training if geometric consistency proves essential for long-horizon generation.
  • The Self-Forcing distillation framework, if it generalizes beyond this model, offers a template for compressing other bidirectional diffusion architectures into causal autoregressive generators without multi-step teacher sampling.
  • NPU-native deployment at 50 FPS would shift the hardware economics of world models: if NPUs already shipping in consumer devices can run these models, the deployment cost advantage over GPU-based solutions could accelerate adoption in consumer-facing applications.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper benchmarks against CameraCtrl, SEVA, WorldPlay, and Lingbot but does not include comparisons against Genie 3 or the undistilled teacher model. If the distilled 4-step student loses significant quality relative to the 50-step teacher, the cost savings may come at a quality tradeoff the paper does not fully surface.
  • The claim of 50 FPS likely depends on specific NPU configurations (multi-NPU deployment with dedicated decoder NPU). Single-NPU performance under the on-demand loading strategy is not quantified at the same FPS, suggesting the 50 FPS figure may require multi-chip setups that narrow the cost advantage.
  • The History Context selection strategy — choosing recent, initial, and camera-related frames — implicitly assumes that viewpoint revisitation is the primary failure mode. For scenes with dynamic object motion rather than camera motion, this selection may be insufficient, which the qualitative results (focused on static environments like corridors and ruins) do not stress-test.
  • If the 3D-native data engine is essential to MoWorld's quality, then reproducibility depends on access to comparable geometric data pipelines, which may be difficult for groups without large-scale 3D vision infrastructure.

Load-bearing premise

The paper assumes that a distribution-matching gradient — the difference between the teacher's estimate of the real data velocity field and a fake model's estimate of the student's velocity field — provides enough corrective signal to preserve visual quality when compressing diffusion sampling from 50 steps to 4 steps in an autoregressive rollout. No ablation isolates the quality impact of this step reduction alone, and the baselines compared against are not the strongest ava

What would settle it

If the 4-step distilled autoregressive student exhibits significant quality degradation, temporal drift, or geometric inconsistency compared to the 50-step teacher model on long-horizon generation (especially beyond the benchmark clip lengths), then the core claim that distillation preserves quality at extreme step compression would be undermined.

read the original abstract

The future of World Models depends not only on scaling model capability, but also on scaling practicality and inference efficiency. High-frame-rate inference enables responsive perception, planning, and control in real-world autonomous systems. To this end, we present MoWorld, a cost-effective yet high-performance Flash World Model with an end-to-end framework spanning data generation, pre-training, distillation, and efficient inference, enabling up to 50 FPS real-time interaction with cinematic visual quality without the need of high-end GPUs. To enable large-scale real-world deployment, MoWorld jointly optimizes model capability and cost throughout the entire development pipeline. Specifically, unlike existing approaches that primarily rely on large-scale video corpora, MoWorld is built upon a scalable 3D-native data engine accumulated from our large-scale 3D vision and generative modeling pipeline, enabling the efficient construction of geometrically consistent training data across diverse real-world and synthetic environments. Based on this foundation, a curriculum cross-frame pre-training strategy for stable and scalable World Model learning, an efficient denoising-step distillation algorithm to reduce diffusion training cost, and a mixed-precision parallel inference framework for low-cost real-time deployment. MoWorld is the first real-time interactive World Model built on the Neural Processing Unit (NPU) and can achieves up to 50 FPS in such the devices, enabling practical and efficient deployment at scale. Comprehensive evaluations demonstrate that MoWorld achieves leading performance; notably, its average inference cost is only 30\%-50\% of that of existing World Models, providing a practical foundation for large-scale real-world applications of World Models. We also demonstrate diverse applications of MoWorld.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 11 minor

Summary. The paper presents MoWorld, a 'Flash World Model' designed for real-time interactive world simulation on NPUs. The system integrates four components: (1) a geometry-aware data engine building on the authors' prior VGGT/VGGT-Omega work, (2) curriculum cross-frame pretraining extending the Wan2.2-A14B backbone with camera control via Plücker embeddings, (3) autoregressive distillation compressing diffusion sampling from 50 to 4 steps using a Self-Forcing/DMD-style approach with history context selection, and (4) NPU-specific inference optimizations including on-demand module loading, hierarchical sequence parallelism, INT8 quantization, and fused attention kernels. The paper claims up to 50 FPS inference and 30-50% cost reduction versus existing world models, and reports VBench-I2V quality metrics against CameraCtrl, SEVA, Lingbot, and WorldPlay. Downstream applications (video transfer, editing, point cloud reconstruction, 3DGS, navigation) are demonstrated qualitatively.

Significance. The paper's end-to-end co-design approach spanning data, training, distillation, and NPU deployment is ambitious and addresses a practical gap in world model accessibility. The curriculum training strategy (125F→2000F) and the ODE-initialization-free distillation design (Sec. 4.4) are reasonable engineering contributions. The history context selection mechanism (Sec. 4.2) with camera-related frame retrieval is a thoughtful design for long-horizon consistency. The VBench-I2V results (Tables 1-2) show competitive quality metrics. However, the paper's central differentiating claims—real-time 50 FPS inference and 30-50% cost reduction—lack quantitative experimental support, which significantly undermines the contribution's verifiability.

major comments (3)
  1. The headline claims of 'up to 50 FPS' (Abstract, Sec. 1, Sec. 5) and 'average inference cost is only 30%-50% of existing World Models' (Abstract, Sec. 1, Sec. 8) have no dedicated experimental measurement anywhere in the paper. Section 5 describes optimization techniques (on-demand module loading, hierarchical sequence parallelism, INT8 quantization, fused attention kernels) qualitatively, but Section 6 contains only VBench-I2V quality metrics (Tables 1-2). There is no table reporting measured FPS, end-to-end latency, throughput at specific resolutions, or cost comparison against any baseline. Without a performance table specifying NPU model, count/configuration, batch size, resolution, and latency breakdown, these claims are unsubstantiated. This is load-bearing because the paper's core positioning as a 'Flash World Model' depends entirely on the speed/cost claims. A dedicated benchmark
  2. Tables 1-2 compare MoWorld against CameraCtrl, SEVA, Lingbot, and WorldPlay, but do not include comparison against the undistilled teacher model (50-step) or stronger contemporaneous baselines such as Genie 3 [24] or Matrix-Game 2.0 [38], both of which are cited in the paper. Since the central distillation contribution is compressing 50 steps to 4 while preserving quality, an ablation isolating the quality impact of step reduction alone (teacher vs. student) is essential to validate that the Self-Forcing distillation (Sec. 4.4, Eqs. 13-18) does not degrade output quality. Without this, it is unclear whether the 4-step student genuinely preserves teacher quality or whether the competitive metrics in Tables 1-2 reflect the pretrained backbone's strength rather than the distillation pipeline's effectiveness.
  3. The distribution-matching gradient g_DMD = v_real - v_fake (Eq. 17) is the core signal correcting rollout errors during AR generation under extreme step compression (50→4 steps). However, no ablation is provided isolating the contribution of the distribution-matching loss (Eq. 18) versus the AR Flow Matching pretraining loss (Eq. 12) alone. Given that the paper claims the distillation 'enables correction of rollout errors while compressing generation into few-step inference' (Sec. 4.4), empirical evidence that the DMD gradient provides meaningful correction signal under 4-step generation would strengthen the claim. At minimum, a quality comparison of student-with-DMD vs. student-without-DMD would address this.
minor comments (11)
  1. Abstract: 'can achieves up to 50 FPS in such the devices' — grammatical error; should be 'can achieve up to 50 FPS on such devices' or similar.
  2. Sec. 3.2 heading: 'Cameral Control' should be 'Camera Control'.
  3. Sec. 4.3.2 heading: 'Autogressive' should be 'Autoregressive'.
  4. Sec. 5.1: 'To enable real-time prompt human interaction' — likely should be 'real-time prompt human interaction' → 'real-time, prompt human interaction' or 'prompt, real-time human interaction'; the intended meaning is unclear.
  5. Sec. 5.2: 'x: left and y: middle of the Figure 5' — the coordinate notation for figure subpanels is unconventional and confusing; standard subfigure labels (a), (b), (c) would be clearer.
  6. Sec. 6.1: 'aTo visually assess' — stray character 'a' before the paragraph.
  7. Table 2: MoWorld's Quality score (86.54) is lower than WorldPlay's (83.04) and Lingbot's (83.66) in the 'Quality' column, yet MoWorld is not bolded. However, the Average score (90.31) is the highest and is bolded. The Quality column formatting should be checked for consistency with the stated bold/underline convention.
  8. Sec. 2: 'curate the large-scale' should be 'curates large-scale' for subject-verb agreement.
  9. Sec. 1: 'existing popular solutions for world models [24, 25, 38, 38]' — reference [38] is duplicated; should likely be [24, 25, 38] or a different second reference.
  10. The paper would benefit from a clear specification of the NPU hardware used (model, memory, count) in a dedicated experimental setup subsection, particularly given the NPU-centric claims.
  11. Eqs. (13)-(18): notation is consistent but the relationship between the Fake Model (Sec. 4.4) and the student model could be stated more explicitly — are they architecturally identical? This would help readers understand the training dynamics.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the careful and constructive review. The referee correctly identifies that our headline speed/cost claims lack dedicated quantitative experimental support in the current manuscript, and we agree this must be addressed. We also agree that teacher-vs-student ablations and DMD loss ablations would strengthen the paper. We will incorporate all three major points in the revision.

read point-by-point responses
  1. Referee: The headline claims of 'up to 50 FPS' and '30%-50% cost reduction' have no dedicated experimental measurement anywhere in the paper. Section 5 describes optimization techniques qualitatively, but Section 6 contains only VBench-I2V quality metrics. Without a performance table specifying NPU model, count/configuration, batch size, resolution, and latency breakdown, these claims are unsubstantiated.

    Authors: The referee is correct. The current manuscript states the 50 FPS and 30-50% cost reduction claims in the Abstract, Section 1, Section 5, and Section 8, but does not provide a dedicated performance benchmark table in Section 6 to substantiate them. This is a genuine gap. We will add a new performance evaluation table reporting: (1) NPU model and configuration (single-NPU and multi-NPU setups on Huawei Ascend), (2) resolution and batch size, (3) end-to-end latency breakdown (encoding, DiT denoising, VAE decoding), (4) measured FPS under both single-NPU and multi-NPU configurations, and (5) cost comparison against representative baselines (CameraCtrl, SEVA, Lingbot, WorldPlay) measured under comparable settings. We will also add per-component latency contributions from each optimization (on-demand module loading, hierarchical sequence parallelism, INT8 quantization, fused attention kernels) via an ablation table. If, upon final measurement, any claim cannot be fully supported at the stated level, we will revise the claim accordingly. revision: yes

  2. Referee: Tables 1-2 do not include comparison against the undistilled teacher model (50-step) or stronger contemporaneous baselines such as Genie 3 or Matrix-Game 2.0. An ablation isolating the quality impact of step reduction alone (teacher vs. student) is essential to validate that the Self-Forcing distillation does not degrade output quality.

    Authors: We agree that a teacher-vs-student ablation is essential to validate the distillation pipeline. In the revised manuscript, we will add a comparison between the 50-step teacher model and the 4-step distilled student on the same VBench-I2V evaluation protocol, isolating the quality impact of step compression. This will directly show whether the Self-Forcing/DMD distillation preserves teacher quality. Regarding Genie 3 [24] and Matrix-Game 2.0 [38]: we cite these works but note that Genie 3 is a closed system without publicly available checkpoints or standardized evaluation protocol matching ours, and Matrix-Game 2.0, while open-source, operates under a different interactive streaming formulation that makes direct VBench-I2V comparison not straightforward. We will include Matrix-Game 2.0 if a fair evaluation setup can be established; if not, we will explicitly discuss the methodological differences that prevent direct comparison and clarify this in the revision. revision: partial

  3. Referee: No ablation isolating the contribution of the distribution-matching loss (Eq. 18) versus the AR Flow Matching pretraining loss (Eq. 12) alone. Empirical evidence that the DMD gradient provides meaningful correction signal under 4-step generation would strengthen the claim.

    Authors: This is a fair request. We will add an ablation comparing: (a) student trained with AR Flow Matching pretraining only (Eq. 12, no DMD loss), versus (b) student trained with the full Self-Forcing distillation including the distribution-matching gradient (Eq. 17-18). Both will be evaluated at 4-step inference on the same VBench-I2V metrics. This will directly demonstrate whether the DMD gradient provides meaningful correction signal under extreme step compression. We expect this ablation to show that AR Flow Matching pretraining alone produces a functional but lower-quality 4-step student, while the DMD distribution-matching signal measurably improves quality by correcting rollout errors that accumulate during autoregressive generation. revision: yes

Circularity Check

0 steps flagged

No significant circularity; self-citations are to independently published methods, and the headline FPS/cost claims are unsupported but not circular.

full rationale

The paper's derivation chain is largely self-contained. The core technical contributions—curriculum cross-frame pretraining (Eq. 1), AR Flow Matching (Eqs. 7-12), and Self-Forcing distillation (Eqs. 13-18)—are derived from standard diffusion and flow-matching objectives without reducing to their own inputs by construction. The distillation gradient g_DMD = v_real - v_fake (Eq. 17) follows the DMD framework [61], which is an externally published method, not a self-citation. The data engine reuses VGGT-Omega [3] for geometry construction, but this is a reuse of prior work (acknowledged explicitly: 'Our protocol shares the design of VGGT-Omega'), not a circular derivation. The backbone Wan2.2 [42] is an open-source external model. The team's own prior work (VGGT [2], VGGT-Omega [3], HD-VGGT [41], StreamCacheVGGT [40]) is cited for the data pipeline and SfM components, but these are tool reuse, not load-bearing theoretical claims that would be circular. The Self-Forcing distillation design (Sec. 4.4) is adapted from [59] (Self-Forcing) and [61] (DMD), both external. The one concern is that the headline claims of 'up to 50 FPS' and '30-50% inference cost' (Abstract, Sec. 1, Sec. 5) have no quantitative experimental support in the paper—Section 6 reports only VBench-I2V quality metrics (Tables 1-2) with no FPS, latency, or cost measurements. This is a significant evidentiary gap (an unsupported claim), but it is not circularity: the claim is not defined in terms of itself or derived from a self-citation chain. The optimization techniques described in Section 5 (on-demand loading, hierarchical sequence parallelism, INT8 quantization, fused attention) are standard system-level techniques, not ansatz smuggled through self-citation. Overall, the derivation is self-contained against external benchmarks and methods, with no step that reduces to its inputs by construction.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 0 invented entities

The paper introduces no new particles, forces, dimensions, or postulated entities. All architectural components (DiT, MoE, VAE, Plücker coordinates, RoPE) are standard. The 'Flash World Model' category is a naming convention, not a new entity. The 'History Bank' (Sec. 4.2) is a data structure, not a postulated physical or mathematical object. The free parameters are engineering hyperparameters chosen empirically, not fundamental constants.

free parameters (5)
  • lambda_anchor = 1.0
    Stated in Eq. 12 as the weight for the anchor loss. No justification given for this specific value.
  • Curriculum schedule (125F/250F → 500F/1000F → 2000F) = 125, 250, 500, 1000, 2000 frames
    Sec. 3.3. The frame thresholds for each curriculum stage are chosen by hand without ablation.
  • Distillation steps (50 → 4) = 4 steps
    Sec. 4.1 and Fig. 4. The target number of inference steps is a design choice; no ablation on step count vs. quality is provided.
  • History Context selection parameters = Not specified
    Sec. 4.2 describes recent frames, initial frame, and camera-related frames as selected, but the number of each type and the distance threshold for camera-related retrieval are not stated.
  • INT8 quantization scope = DiT blocks only; encoders in BF16
    Sec. 5.4. The decision to quantize only DiT blocks is a design choice justified by encoder sensitivity but not ablated.
axioms (5)
  • domain assumption Wan2.2-A14B [42] is a suitable backbone for camera-controllable world modeling
    Sec. 3.1. The paper builds directly on Wan2.2 without justifying why this backbone is preferred over alternatives for the world-model setting.
  • domain assumption Plücker camera coordinates provide sufficient geometric conditioning for world simulation
    Sec. 3.2, Eqs. 3-5. Inherited from CameraCtrl [49] and SEVA [55]; assumed without comparison to alternative camera representations.
  • domain assumption The distribution-matching gradient (v_real - v_fake) from DMD-style distillation transfers to the autoregressive rollout setting without modification
    Sec. 4.4, Eqs. 16-18. The paper applies DMD-style distillation to AR rollouts, assuming the proxy gradient remains valid under chunk-wise autoregressive generation.
  • domain assumption 3D-native geometrically consistent training data is superior to web-scraped video for world model training
    Sec. 2. The paper argues for this but provides no controlled comparison between 3D-native and web-scraped data.
  • domain assumption VBench-I2V metrics adequately measure world-model quality
    Sec. 6.1. The paper uses VBench-I2V as the primary quantitative benchmark, but these metrics measure video generation quality, not world-model fidelity (e.g., action-response accuracy, physical plausibility).

pith-pipeline@v1.1.0-glm · 24567 in / 3632 out tokens · 477602 ms · 2026-07-08T12:55:02.936140+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

77 extracted references · 77 canonical work pages · 40 internal anchors

  1. [1]

    GPT-4 Technical Report

    OpenAI. GPT-4 technical report.arXiv preprint arXiv:2303.08774, 2023

  2. [2]

    Vggt: Visual geometry grounded transformer

    Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. Vggt: Visual geometry grounded transformer. 2025

  3. [3]

    VGGT-$\Omega$

    Jianyuan Wang, Minghao Chen, Shangzhan Zhang, Nikita Karaev, Johannes Schönberger, Patrick Labatut, Piotr Bojanowski, David Novotny, Andrea Vedaldi, and Christian Rupprecht. Vggt-omega. arXiv preprint arXiv:2605.15195, 2026

  4. [4]

    Seedance 1.5 pro: A Native Audio-Visual Joint Generation Foundation Model

    Seedance Team. Seedance 1.5 pro: A native audio-visual joint generation foundation model. arXiv preprint arXiv:2512.13507, 2025

  5. [5]

    Longcat-video technical report.arXiv preprint arXiv:2510.22200, 2025

    Meituan LongCat Team. Longcat-video technical report.arXiv preprint arXiv:2510.22200, 2025

  6. [6]

    Dens3R: A Foundation Model for 3D Geometry Prediction

    Xianze Fang, Jingnan Gao, Zhe Wang, Zhuo Chen, Xingyu Ren, Jiangjing Lyu, Qiaomu Ren, Zhonglei Yang, Xiaokang Yang, Yichao Yan, and Chengfei Lyu. Dens3r: A foundation model for 3d geometry prediction.arXiv preprint arXiv:2507.16290, 2025

  7. [9]

    Veo 3.1, 2025

    Google DeepMind. Veo 3.1, 2025. URLhttps://deepmind.google/technologies/veo/

  8. [10]

    Kling 2.5 turbo, 2025

    Kuaishou Technology. Kling 2.5 turbo, 2025. URLhttps://app.klingai.com/cn/release-notes/2025-09-19

  9. [11]

    Sora 2, 2025

    OpenAI. Sora 2, 2025. URLhttps://openai.com/sora

  10. [12]

    HunyuanVideo: A Systematic Framework For Large Video Generative Models

    Tencent Hunyuan Foundation Model Team. Hunyuanvideo: A systematic framework for large video generative models. arXiv preprint arXiv:2412.03603, 2025. URLhttps://arxiv.org/abs/2412.03603

  11. [13]

    Cosmos-Drive-Dreams: Scalable Synthetic Driving Data Generation with World Foundation Models

    Xuanchi Ren, Yifan Lu, Tianshi Cao, Ruiyuan Gao, Shengyu Huang, Amirmojtaba Sabour, Tianchang Shen, Tobias Pfaff, Jay Zhangjie Wu, Runjian Chen, Seung Wook Kim, Jun Gao, Laura Leal-Taixe, Mike Chen, Sanja Fidler, and Huan Ling. Cosmos-drive-dreams: Scalable synthetic driving data generation with world foundation models. arXiv preprint arXiv:2506.09042, 2025

  12. [14]

    Diffusion Models Are Real-Time Game Engines

    Dani Valevski, Yaniv Leviathan, Moab Arar, and Shlomi Fruchter. Diffusion models are real-time game engines. arXiv preprint arXiv:2408.14837, 2024

  13. [15]

    DeepSeek-VL: Towards Real-World Vision-Language Understanding

    Haoyu Lu, Wen Liu, Bo Zhang, Bingxuan Wang, Kai Dong, Bo Liu, Jingxiang Sun, Tongzheng Ren, Zhuoshu Li, Hao Yang, Yaofeng Sun, Chengqi Deng, Hanwei Xu, Zhenda Xie, and Chong Ruan. Deepseek-vl: towards real-world vision-language understanding.arXiv preprint arXiv:2403.05525, 2024

  14. [16]

    ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools

    Team Glm, Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin, Dan Zhang, Diego Rojas, Guanyu Feng, Hanlin Zhao, et al. Chatglm: A family of large language models from glm-130b to glm-4 all tools.arXiv preprint arXiv:2406.12793, 2024

  15. [17]

    DeepSeek-V3 Technical Report

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437, 2024

  16. [18]

    Qwen3-VL Technical Report

    Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, et al. Qwen3-vl technical report.arXiv preprint arXiv:2511.21631, 2025

  17. [19]

    Dong Guo, Faming Wu, Feida Zhu, Fuxing Leng, Guang Shi, Haobin Chen, Haoqi Fan, Jian Wang, Jianyu Jiang, Jiawei Wang, et al. Seed1. 5-vl technical report.arXiv preprint arXiv:2505.07062, 2025

  18. [20]

    Gemini: A Family of Highly Capable Multimodal Models

    Google Gemini Team. Gemini: a family of highly capable multimodal models.arXiv preprint arXiv:2312.11805, 2023

  19. [21]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. 2021. 26

  20. [22]

    Hunyuan-gamecraft-2: Instruction-following interactive game world model.arXiv preprint arXiv:2511.23429, 2025

    Junshu Tang, Jiacheng Liu, Jiaqi Li, Longhuang Wu, Haoyu Yang, Penghao Zhao, Siruis Gong, Xiang Yuan, Shuai Shao, and Qinglin Lu. Hunyuan-gamecraft-2: Instruction-following interactive game world model.arXiv preprint arXiv:2511.23429, 2025

  21. [23]

    Pan: A world model for general, interactable, and long-horizon world simulation.arXiv preprint arXiv:2511.09057, 2025

    PAN Team. Pan: A world model for general, interactable, and long-horizon world simulation.arXiv preprint arXiv:2511.09057, 2025

  22. [24]

    Philip J. Ball, Jakob Bauer, Frank Belletti, Bethanie Brownfield, Ariel Ephrat, Shlomi Fruchter, Agrim Gupta, Kristian Holsheimer, Aleksander Holynski, Jiri Hron, Christos Kaplanis, Marjorie Limont, Matt McGill, Yanko Oliveira, Jack Parker-Holder, Frank Perbet, Guy Scully, Jeremy Shar, Stephen Spencer, Omer Tov, Ruben Villegas, Emma Wang, Jessica Yung, Ci...

  23. [25]

    Advancing Open-source World Models

    Robbyant Team, Zelin Gao, Qiuyu Wang, Yanhong Zeng, Jiapeng Zhu, Ka Leong Cheng, Yixuan Li, Hanlin Wang, Yinghao Xu, Shuailei Ma, Yihang Chen, Jie Liu, Yansong Cheng, Yao Yao, Jiayi Zhu, Yihao Meng, Kecheng Zheng, Qingyan Bai, Jingye Chen, Zehong Shen, Yue Yu, Xing Zhu, Yujun Shen, and Hao Ouyang. Advancing open-source world models.arXiv preprint arXiv:26...

  24. [26]

    Hy-world 1.5: A systematic framework for interactive world modeling with real-time latency and geometric consistency.arXiv preprint, 2025

    Team HunyuanWorld. Hy-world 1.5: A systematic framework for interactive world modeling with real-time latency and geometric consistency.arXiv preprint, 2025

  25. [27]

    WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modeling

    Wenqiang Sun, Haiyu Zhang, Haoyuan Wang, Junta Wu, Zehan Wang, Zhenwei Wang, Yunhong Wang, Jun Zhang, Tengfei Wang, and Chunchao Guo. Worldplay: Towards long-term geometric consistency for real-time interactive world modeling.arXiv preprint arXiv:2512.14614, 2025

  26. [28]

    Yume-1.5: A text-controlled interactive world generation model.arXiv preprint arXiv:2512.22096, 2025

    XiaofengMao, ZhenLi, ChuanhaoLi, XiaojieXu, KainingYing, TongHe, JiangmiaoPang, YuQiao, andKaipeng Zhang. Yume-1.5: A text-controlled interactive world generation model.arXiv preprint arXiv:2512.22096, 2025

  27. [29]

    World Simulation with Video Foundation Models for Physical AI

    Arslan Ali, Junjie Bai, Maciej Bala, Yogesh Balaji, Aaron Blakeman, Tiffany Cai, Jiaxin Cao, Tianshi Cao, Elizabeth Cha, Yu-Wei Chao, et al. World simulation with video foundation models for physical ai. arXiv preprint arXiv:2511.00062, 2025

  28. [30]

    TesserAct: Learning 4D Embodied World Models

    Haoyu Zhen, Qiao Sun, Hongxin Zhang, Junyan Li, Siyuan Zhou, Yilun Du, and Chuang Gan. Tesseract: learning 4d embodied world models.arXiv preprint arXiv:2504.20995, 2025

  29. [31]

    The world is your canvas: Paintingpromptableeventswithreferenceimages, trajectories, andtext

    Hanlin Wang, Hao Ouyang, Qiuyu Wang, Yue Yu, Yihao Meng, Wen Wang, Ka Leong Cheng, Shuailei Ma, Qingyan Bai, Yixuan Li, Cheng Chen, Yanhong Zeng, Xing Zhu, Yujun Shen, and Qifeng Chen. The world is your canvas: Paintingpromptableeventswithreferenceimages, trajectories, andtext. arXivpreprintarXiv:2512.16924, 2025

  30. [32]

    World Action Models are Zero-shot Policies

    Seonghyeon Ye, Yunhao Ge, Kaiyuan Zheng, Shenyuan Gao, Sihyun Yu, George Kurian, Suneel Indupuru, You Liang Tan, Chuning Zhu, Jiannan Xiang, et al. World action models are zero-shot policies.arXiv preprint arXiv:2602.15922, 2026

  31. [33]

    Motus: Aunifiedlatentactionworldmodel

    Hongzhe Bi, Hengkai Tan, Shenghao Xie, Zeyuan Wang, Shuhe Huang, Haitian Liu, Ruowen Zhao, Yao Feng, ChendongXiang, YinzeRong, etal. Motus: Aunifiedlatentactionworldmodel. In Proceedings oftheIEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 35101–35113, 2026

  32. [34]

    World-Value-Action Model: Implicit Planning for Vision-Language-Action Systems

    Runze Li, Hongyin Zhang, Junxi Jin, Qixin Zeng, Zifeng Zhuang, Yiqi Tang, Shangke Lyu, and Donglin Wang. World-value-action model: Implicit planning for vision-language-action systems.arXivpreprintarXiv:2604.14732, 2026

  33. [35]

    Learning physics from pretrained video models: A multimodal continuous and sequential world interaction models for robotic manipulation

    Zijian Song, Qichang Li, Sihan Qin, Yuhao Chen, Tianshui Chen, Liang Lin, and Guangrun Wang. Learning physics from pretrained video models: A multimodal continuous and sequential world interaction models for robotic manipulation. InProceedings of the 2026 International Conference on Multimedia Retrieval, pages 758– 767, 2026

  34. [36]

    Cosmos World Foundation Model Platform for Physical AI

    NVIDIA. Cosmos world foundation model platform for physical ai.arXiv preprint arXiv:2501.03575, 2025. 27

  35. [37]

    A Survey: Learning Embodied Intelligence from Physical Simulators and World Models

    Xiaoxiao Long, Qingrui Zhao, Kaiwen Zhang, Zihao Zhang, Dingrui Wang, Yumeng Liu, Zhengjie Shu, Yi Lu, Shouzheng Wang, Xinzhe Wei, Wei Li, Wei Yin, Yao Yao, Jia Pan, Qiu Shen, Ruigang Yang, Xun Cao, and Qionghai Dai. A survey: Learning embodied intelligence from physical simulators and world models. arXiv preprint arXiv:2507.00917, 2025

  36. [38]

    Matrix-game 2.0: An open-source real-time and streaming interactive world model

    Xianglong He, Chunli Peng, Zexiang Liu, Boyang Wang, Yifan Zhang, Qi Cui, Fei Kang, Biao Jiang, Mengyin An, Yangyang Ren, Baixin Xu, Hao-Xiang Guo, Kaixiong Gong, Size Wu, Wei Li, Xuchen Song, Yang Liu, Yangguang Li, and Yahui Zhou. Matrix-game 2.0: An open-source real-time and streaming interactive world model. arXiv preprint arXiv:2508.13009, 2025

  37. [39]

    Vggsfm: Visual geometry grounded deep structure from motion

    Jianyuan Wang, Nikita Karaev, Christian Rupprecht, and David Novotny. Vggsfm: Visual geometry grounded deep structure from motion. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 21686–21697, 2024

  38. [40]

    StreamCacheVGGT: Streaming Visual Geometry Transformers with Robust Scoring and Hybrid Cache Compression

    Xuanyi Liu, Chunan Yu, Deyi Ji, Qi Zhu, Lingyun Sun, Xuanfu Li, Jin Ma, Tianrun Chen, and Lanyun Zhu. Streamcachevggt: Streaming visual geometry transformers with robust scoring and hybrid cache compression. arXiv preprint arXiv:2604.15237, 2026

  39. [41]

    HD-VGGT: High-Resolution Visual Geometry Transformer

    Tianrun Chen, Yuanqi Hu, Yidong Han, Hanjie Xu, Deyi Ji, Qi Zhu, Chunan Yu, Xin Zhang, Cheng Chen, Chaotao Ding, et al. Hd-vggt: High-resolution visual geometry transformer.arXiv preprint arXiv:2603.27222, 2026

  40. [42]

    Wan: Open and Advanced Large-Scale Video Generative Models

    Wan Team. Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314, 2025

  41. [43]

    Denoising diffusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020

  42. [44]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022

  43. [45]

    Align your latents: High-resolution video synthesis with latent diffusion models

    Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dockhorn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with latent diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 22563–22575, 2023

  44. [46]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. InProceedings of the IEEE/CVF international conference on computer vision, pages 4195–4205, 2023

  45. [47]

    A survey on mixture of experts in large language models.IEEE Transactionson Knowledge and Data Engineering, 2025

    Weilin Cai, Juyong Jiang, Fan Wang, Jing Tang, Sunghun Kim, and Jiayi Huang. A survey on mixture of experts in large language models.IEEE Transactionson Knowledge and Data Engineering, 2025

  46. [48]

    More: 3d visual geometry reconstruction meets mixture-of-experts

    Jingnan Gao, Zhe Wang, Xianze Fang, Xingyu Ren, Zhuo Chen, Shengqi Liu, Yuhao Cheng, Jiangjing Lyu, Xiaokang Yang, and Yichao Yan. More: 3d visual geometry reconstruction meets mixture-of-experts. arXiv preprint arXiv:2510.27234, 2025

  47. [49]

    CameraCtrl: Enabling Camera Control for Text-to-Video Generation

    Hao He, Yinghao Xu, Yuwei Guo, Gordon Wetzstein, Bo Dai, Hongsheng Li, and Ceyuan Yang. Cameractrl: Enabling camera control for text-to-video generation.arXiv preprint arXiv:2404.02101, 2024

  48. [50]

    CamI2V: Camera-Controlled Image-to-Video Diffusion Model

    Guangcong Zheng, Teng Li, Rui Jiang, Yehao Lu, Tao Wu, and Xi Li. Cami2v: Camera-controlled image-to-video diffusion model. arXiv preprint arXiv:2410.15957, 2024

  49. [51]

    VD3D: Taming Large Video Diffusion Transformers for 3D Camera Control

    Sherwin Bahmani, Ivan Skorokhodov, Aliaksandr Siarohin, Willi Menapace, Guocheng Qian, Michael Vasilkovsky, Hsin-Ying Lee, Chaoyang Wang, Jiaxu Zou, Andrea Tagliasacchi, et al. Vd3d: Taming large video diffusion transformers for 3d camera control.arXiv preprint arXiv:2407.12781, 2024

  50. [52]

    CamCo: Camera-Controllable 3D-Consistent Image-to-Video Generation

    Dejia Xu, Weili Nie, Chao Liu, Sifei Liu, Jan Kautz, Zhangyang Wang, and Arash Vahdat. Camco: Camera- controllable 3d-consistent image-to-video generation.arXiv preprint arXiv:2406.02509, 2024

  51. [53]

    CPA: Camera-pose-awareness Diffusion Transformer for Video Generation

    Yuelei Wang, Jian Zhang, Pengtao Jiang, Hao Zhang, Jinwei Chen, and Bo Li. Cpa: Camera-pose-awareness diffusion transformer for video generation.arXiv preprint arXiv:2412.01429, 2024

  52. [54]

    CamGeo: Sparse Camera-Conditioned Image-to-Video Generation with 3D Geometry Priors

    Xuanyi Liu, Deyi Ji, Liqun Liu, Lanyun Zhu, Xuhang Chen, Qianxiong Xu, Peng Shu, Huan Yu, Jie Jiang, Feng Gao, et al. Camgeo: Sparse camera-conditioned image-to-video generation with 3d geometry priors.arXiv preprint arXiv:2605.30895, 2026. 28

  53. [55]

    Stable virtual camera: Generative view synthesis with diffusion models

    Jensen Zhou, Hang Gao, Vikram Voleti, Aaryaman Vasishta, Chun-Han Yao, Mark Boss, Philip Torr, Christian Rupprecht, and Varun Jampani. Stable virtual camera: Generative view synthesis with diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 12405–12414, 2025

  54. [56]

    USP: A Unified Sequence Parallelism Approach for Long Context Generative AI

    J. Fang and S. Zhao. Usp: A unified sequence parallelism approach for long context generative ai.arXiv preprint arXiv:2405.07719, 2024

  55. [57]

    DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Models

    Sam Ade Jacobs, Masahiro Tanaka, Chengming Zhang, Minjia Zhang, Shuaiwen Leon Song, Samyam Rajbhan- dari, and Yuxiong He. Deepspeed ulysses: System optimizations for enabling training of extreme long sequence transformer models. arXiv preprint arXiv:2309.14509, 2023

  56. [58]

    Zero: Memory optimizations toward train- ing trillion parameter models

    Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. Zero: Memory optimizations toward train- ing trillion parameter models. InProceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC20), pages 1–16. IEEE, 2020

  57. [59]

    Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion

    Xun Huang, Zhengqi Li, Guande He, Mingyuan Zhou, and Eli Shechtman. Self forcing: Bridging the train-test gap in autoregressive video diffusion.arXiv preprint arXiv:2506.08009, 2025

  58. [60]

    Self-Forcing++: Towards Minute-Scale High-Quality Video Generation

    Justin Cui, Jie Wu, Ming Li, Tao Yang, Xiaojie Li, Rui Wang, Andrew Bai, Yuanhao Ban, and Cho-Jui Hsieh. Self-forcing++: Towards minute-scale high-quality video generation.arXiv preprint arXiv:2510.02283, 2025

  59. [61]

    One-step diffusion with distribution matching distillation

    Tianwei Yin, Michaël Gharbi, Richard Zhang, Eli Shechtman, Frédo Durand, William T Freeman, and Taesung Park. One-step diffusion with distribution matching distillation. 2024

  60. [62]

    Improved distribution matching distillation for fast image synthesis

    Tianwei Yin, Michaël Gharbi, Taesung Park, Richard Zhang, Eli Shechtman, Fredo Durand, and Bill Freeman. Improved distribution matching distillation for fast image synthesis. 2024

  61. [63]

    From slow bidirectional to fast autoregressive video diffusion models

    Tianwei Yin, Qiang Zhang, Richard Zhang, William T Freeman, Fredo Durand, Eli Shechtman, and Xun Huang. From slow bidirectional to fast autoregressive video diffusion models. InCVPR, 2025

  62. [64]

    Distilling the Knowledge in a Neural Network

    Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network.arXiv preprint arXiv:1503.02531, 2015

  63. [65]

    MAGI-1: Autoregressive Video Generation at Scale

    Sand.ai. Magi-1: Autoregressive video generation at scale.arXiv preprint arXiv:2505.13211, 2025

  64. [66]

    Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generation

    Hongzhou Zhu, Min Zhao, Guande He, Hang Su, Chongxuan Li, and Jun Zhu. Causal forcing: Autoregres- sive diffusion distillation done right for high-quality real-time interactive video generation. arXiv preprint arXiv:2602.02214, 2026

  65. [67]

    Worldmem: Long-term consistent world simulation with memory.Advances in Neural Information Processing Systems, 38: 49632–49652, 2026

    Zeqi Xiao, Yushi Lan, Yifan Zhou, Wenqi Ouyang, Shuai Yang, Yanhong Zeng, and Xingang Pan. Worldmem: Long-term consistent world simulation with memory.Advances in Neural Information Processing Systems, 38: 49632–49652, 2026

  66. [68]

    CAT3D: Create Anything in 3D with Multi-View Diffusion Models

    Ruiqi Gao, Aleksander Holynski, Philipp Henzler, Arthur Brussee, Ricardo Martin-Brualla, Pratul Srinivasan, Jonathan T Barron, and Ben Poole. Cat3d: Create anything in 3d with multi-view diffusion models. arXiv preprint arXiv:2405.10314, 2024

  67. [69]

    Reconx: Reconstruct any scene from sparse views with video diffusion model

    Fangfu Liu, Wenqiang Sun, Hanyang Wang, Yikai Wang, Haowen Sun, Junliang Ye, Jun Zhang, and Yueqi Duan. Reconx: Reconstruct any scene from sparse views with video diffusion model. IEEE Transactions on Image Processing, 2026

  68. [70]

    Gen3c: 3d-informed world-consistent video generation with precise camera control

    Xuanchi Ren, Tianchang Shen, Jiahui Huang, Huan Ling, Yifan Lu, Merlin Nimier-David, Thomas Müller, Alexander Keller, Sanja Fidler, and Jun Gao. Gen3c: 3d-informed world-consistent video generation with precise camera control. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6121–6132, 2025

  69. [71]

    Roformer: Enhanced trans- former with rotary position embedding.Neurocomputing, 568:127063, 2024

    Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced trans- former with rotary position embedding.Neurocomputing, 568:127063, 2024

  70. [72]

    Helios: Real real-time long video generation model.arXiv preprint arXiv:2603.04379, 2026

    Shenghai Yuan, Yuanyang Yin, Zongjian Li, Xinwei Huang, Xiao Yang, and Li Yuan. Helios: Real real-time long video generation model.arXiv preprint arXiv:2603.04379, 2026

  71. [73]

    Flow Matching for Generative Modeling

    Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXiv:2210.02747, 2022

  72. [74]

    Ringattention with blockwise transformers for near-infinite context

    Hao Liu, Matei Zaharia, and Pieter Abbeel. Ringattention with blockwise transformers for near-infinite context. In International Conference on Learning Representations, volume 2024, pages 3992–4008, 2024. 29

  73. [75]

    World model on million-length video and language with blockwise ringattention

    Hao Liu, Wilson Yan, Matei Zaharia, and Pieter Abbeel. World model on million-length video and language with blockwise ringattention. In International Conference on Learning Representations, volume 2025, pages 45953–45977, 2025

  74. [76]

    VBench: Comprehensive benchmark suite for video generative models

    Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, Yaohui Wang, Xinyuan Chen, Limin Wang, Dahua Lin, Yu Qiao, and Ziwei Liu. VBench: Comprehensive benchmark suite for video generative models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024

  75. [77]

    InternVLA-N1: An open dual-system navigation foundation model with learned latent plans, 2025

    InternNav Team. InternVLA-N1: An open dual-system navigation foundation model with learned latent plans, 2025

  76. [78]

    3d gaussian splatting for real-time radiance field rendering.ACM Trans

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, George Drettakis, et al. 3d gaussian splatting for real-time radiance field rendering.ACM Trans. Graph., 42(4):139–1, 2023

  77. [79]

    Feature 3dgs: Supercharging 3d gaussian splatting to enable distilled feature fields

    Shijie Zhou, Haoran Chang, Sicheng Jiang, Zhiwen Fan, Zehao Zhu, Dejia Xu, Pradyumna Chari, Suya You, Zhangyang Wang, and Achuta Kadambi. Feature 3dgs: Supercharging 3d gaussian splatting to enable distilled feature fields. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 21676–21685, 2024. 30