The reviewed record of science sign in
Pith

arxiv: 2607.06291 · v1 · pith:GS5TLAMH · submitted 2026-07-07 · cs.CV · cs.HC

AlayaWorld: Long-Horizon and Playable Video World Generation

AlayaWorld Team , Kaipeng Zhang , Chuanhao Li , Yifan Zhan , Yongtao Ge , Yuanyang Yin , Jiaming Tan , Kang He
show 9 more authors
Liaoyuan Fan Ruicong Liu Xiaojie Xu Xuangeng Chu Zhen Li Zhengyuan Lin Zhixiang Wang Zian Meng Zihui Gao
This is my paper

Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel 2026-07-08 10:42 UTCglm-5.2pith:GS5TLAMHrecord.jsonopen to challenge →

classification cs.CV cs.HC
keywords interactive world modelsvideo generationautoregressive diffusion3D cachelong-horizon videoreal-time generationspatial memoryerror bank
0
0 comments X

The pith

Open-source framework generates playable video worlds in real time

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

AlayaWorld is a full-stack, open-source framework that uses autoregressive video diffusion to synthesize explorable, interactive 3D worlds on the fly, without manually authoring environments. The central claim is that four engineering problems—precise camera control, spatial consistency on revisits, long-horizon stability, and real-time latency—can be jointly solved by combining a 3D scene cache with compressed temporal history, an error-bank training procedure, few-step distillation, and chunk-level prompt switching. Fine-tuned from LTX-2.3, the system generates 720p 24fps video where each one-second chunk is produced in four denoising steps, allowing users to navigate freely, cast spells, fight, and summon monsters while the world persists and remains visually coherent over minute-scale rollouts. The paper frames this as a practical foundation: it releases reproducible pipelines, reference implementations, and evaluation tools alongside the model.

Core claim

The paper's central contribution is the integration of six mechanisms into a single autoregressive DiT pipeline, each targeting a specific failure mode of interactive world generation. A 3D cache (following GEN3C) provides spatially indexed memory so that revisited regions look consistent with their earlier appearance. A lightweight history-compression module (following Frame Preservation) supplies temporal memory for recent dynamics. An error bank injects residual artifacts from past rollouts into both the conditioning memory and the prediction target during training, teaching the model to correct rather than compound its own errors. Few-step distillation reduces per-chunk latency. Short, ~

What carries the argument

Autoregressive DiT (diffusion transformer) fine-tuned from LTX-2.3; 3D cache for spatially indexed memory; compressed temporal history embedding for short-term dynamics; error bank for drift-correction training; AdaLN-style camera modulation; chunk-level prompt switching for real-time action updates; DMD-based few-step distillation for low-latency inference.

If this is right

  • If the approach scales, game worlds and embodied-AI training environments could be generated from video data rather than hand-authored, reducing production cost and enabling rapid content iteration.
  • The combination of spatially indexed 3D cache with temporal compression offers a template for any autoregressive generative system that needs persistent memory over long horizons, not just video worlds.
  • Open-sourcing the full stack—data pipelines, training, inference, deployment—lowers the barrier for independent verification and extension, which matters in a field where the strongest competing systems (e.g., Genie 3) remain closed-source.
  • The error-bank training strategy, if effective, could generalize to other autoregressive generation tasks where self-generated artifacts compound, such as long-form audio or text generation.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper's claims about stability and consistency rest on qualitative figures rather than quantitative metrics; if the error bank and dual-memory approach are as effective as described, measured drift and revisit-consistency benchmarks should confirm this in subsequent releases.
  • The 3D cache primarily captures static structure; the paper acknowledges it cannot represent dynamic object state changes. A natural extension would be a spatiotemporal cache that tracks object-level state evolution, which would be necessary for worlds where NPCs or environmental changes persist meaningfully across long horizons.
  • The chunk-level prompt switching mechanism achieves semantic latency at roughly one-second granularity; sub-second responsiveness would likely require either smaller chunks (increasing per-frame overhead) or a finer-grained conditioning update mechanism.

Load-bearing premise

The paper assumes that the combination of a 3D cache, compressed temporal history, and error-bank training is sufficient to maintain long-horizon stability and spatial consistency, but validates this claim with qualitative figures rather than measured metrics against baselines.

What would settle it

Run AlayaWorld and competing open-source systems (e.g., Hunyuan-GameCraft, Matrix-Game, Yume 1.5) on identical leave-and-return trajectories and minute-scale forward rollouts. If AlayaWorld's revisited regions show measurable inconsistency with their earlier appearance, or if visual quality degrades over a 60-second rollout more than baselines do, the central claim of joint stability and consistency is not supported.

read the original abstract

Game worlds have traditionally been built through labor-intensive production pipelines, making them costly to develop, difficult to customization, and expensive to modify after deployment. Recent advances in video world models offer a fundamentally different paradigm. Rather than explicitly authoring every component of a virtual environment, these models autoregressively synthesize future observations conditioned on the current world state and user interactions, enabling playable worlds to be generated online. Trained on both gameplay recordings and real-world videos, they can capture diverse visual appearances and physical dynamics, opening new opportunities for interactive applications beyond gaming, including embodied intelligence. In this paper, we present \textbf{AlayaWorld}, a full-stack open-source framework for building interactive generative worlds. AlayaWorld enables open-ended real-time interaction, allowing users to freely navigate and perform diverse actions such as combat, spell casting, and monster summoning. The framework unifies the complete development-from data preparation model architecture, model training, inference acceleration, and deployment-within a modular and extensible architecture. Alongside the framework, we release reproducible pipelines, reference implementations, evaluation tools, and comprehensive documentation, establishing a practical foundation for future research and real-time applications of generative world models.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 7 minor

Summary. This paper presents AlayaWorld, a full-stack, open-source framework for interactive generative world modeling. The system is fine-tuned from LTX-2.3 and integrates several mechanisms to address four challenges: control (camera conditioning via AdaLN modulation and a 3D cache), consistency (spatially indexed 3D cache combined with compressed temporal history), stability (error-bank training with drifted histories), and runtime (DMD-based distillation and chunk-level prompt switching). The paper provides a comprehensive review of related work organized by these four axes, then describes the AlayaWorld design, and finally presents qualitative results (Section 4) across camera control, open-ended action, consistency, long-horizon generation, and diverse styles.

Significance. The paper's primary contribution is as a systems/integration effort: it combines multiple existing techniques (3D cache from GEN3C, history compression from Frame Preservation, error recycling from Stable Video Infinity/Helios, and DMD distillation) into a unified, open-source framework. The related work survey (Section 3) is thorough and well-organized, providing a useful taxonomy for the field. The promise of releasing reproducible pipelines, reference implementations, and evaluation tools is a positive for community value. However, the manuscript currently ships without quantitative evaluation, ablation studies, or the codebase, which significantly limits the ability to assess the system's actual performance and the contribution of each module.

major comments (3)
  1. §4: The entire evaluation section is qualitative, consisting only of figures (Figs. 2–6). There are no quantitative metrics (e.g., FID, FVD, camera-following error, consistency scores, latency measurements), no ablation studies isolating the contribution of the 3D cache, error bank, compressed history, or prompt switching, and no quantitative comparison with baselines. The paper states in §1 that 'complete technical details, experimental results, and full codebase will be released in mid-July,' confirming the evaluation is not yet included. Every performance claim—long-horizon stability over one-minute rollouts (§4.4), loop-closure consistency (§4.3), real-time latency (§3.4), and camera-following accuracy (§4.1)—rests on visual inspection of selected figures alone. This is the central load-bearing issue: the paper's claims cannot be assessed or verified without quantitative evidence.
  2. §4.3: The text states 'We also evaluate representative interactive world models under the same setting' and describes their failure modes (visual degradation, inaccurate camera control, inconsistency on revisits). However, no baselines are named, no metrics are reported, and the comparison appears to be purely visual. The claim that 'all baselines are evaluated under the same input conditioning and resolution' (§4) cannot be verified. Without specifying which baselines were tested, under what protocol, and with what measured outcomes, this comparison does not constitute an evaluation.
  3. §3.4: The runtime claim is central to the paper's positioning as a 'real-time' and 'playable' system, yet no latency measurements are provided. The paper states that each chunk corresponds to roughly one second of video with four denoising steps, but does not report wall-clock generation time, end-to-end interaction latency, or throughput. Without these numbers, the real-time claim is unsupported.
minor comments (7)
  1. §3.1.1: The text says 'AlayaWorld combines explicit rendered evidence with lightweight architectural injection' but does not specify the architecture of the AdaLN camera-control module (e.g., input representation, embedding dimension, where in the DiT it is injected). A brief specification would help reproducibility.
  2. §3.3: The error bank is described conceptually but its construction is underspecified. How are residual artifacts measured and stored? What is the size of the error bank? How are samples selected during training? These details are needed to understand the contribution over Stable Video Infinity's Error-Recycling Fine-Tuning.
  3. §3.1.2: The prompt-switching mechanism is described as operating 'at chunk granularity' but the implementation is not specified. Is the text condition simply replaced before generating the next chunk, or is there a transition mechanism? The text says it is 'conceptually aligned with attention-level prompt editing [23, 38]' but does not clarify whether attention manipulation is actually used.
  4. Abstract: 'difficult to customization' should be 'difficult to customize' or 'difficult to customize.'
  5. §1: 'difficult to customization' (same typo as abstract). Also 'AlayaWorld is a full-stack, open-source, and long-term project' — the manuscript should clarify whether the framework is available at submission time or only promised for future release.
  6. §4: 'For fairness, all baselines are evaluated under the same input conditioning and resolution as AlayaWorld whenever their public implementations permit.' This sentence implies a quantitative comparison that does not appear in the paper. It should be removed or backed by actual results.
  7. Figures 2–6 are referenced but not visible in the text provided. The figure captions are minimal; more detailed captions describing the specific scenario, inputs, and expected outcomes would help readers interpret the qualitative results.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the careful and constructive review. The referee's central concern—that the current manuscript lacks quantitative evaluation, ablation studies, and code—is well-taken. We address each major comment below and confirm that the revised manuscript will incorporate the requested quantitative metrics, ablation studies, named baselines, and latency measurements.

read point-by-point responses

  1. Referee: §4: The entire evaluation section is qualitative, consisting only of figures (Figs. 2–6). There are no quantitative metrics, no ablation studies, and no quantitative comparison with baselines. Every performance claim rests on visual inspection of selected figures alone. This is the central load-bearing issue.

    Authors: The referee is correct. The current manuscript is a systems/integration paper whose evaluation is entirely qualitative, and this is a genuine limitation of the submitted version. We will address this comprehensively in the revision. Specifically, we will add: (1) Quantitative metrics including FID/FVD for visual quality, camera-following error (measured as deviation between requested and estimated camera trajectories via COLMAP), temporal consistency scores, and wall-clock latency measurements. (2) Ablation studies isolating each module: 3D cache, error bank, compressed temporal history, prompt switching, and DMD distillation. Each ablation will remove or replace one component and measure the effect on the relevant metrics. (3) Quantitative comparison with named baselines under a standardized protocol. We agree that visual inspection of selected figures is insufficient to substantiate claims of long-horizon stability, loop-closure consistency, real-time latency, and camera-following accuracy. The revised §4 will present these results in tabular form alongside the existing qualitative figures. revision: yes

  2. Referee: §4.3: The text states 'We also evaluate representative interactive world models under the same setting' but no baselines are named, no metrics are reported, and the comparison appears purely visual. The claim that all baselines are evaluated under the same input conditioning and resolution cannot be verified.

    Authors: The referee is correct that the current §4.3 does not name baselines or report metrics, making the comparison unverifiable. We will revise this section to: (1) Name the specific baselines tested, which will include publicly available interactive world models such as Yume 1.5, Matrix-Game, and Hunyuan-GameCraft, selected based on availability of public weights and inference code. (2) Specify the evaluation protocol: identical input conditioning (same initial frames, camera trajectories, and text prompts), resolution (720p), and chunk structure wherever the baseline architecture permits. (3) Report quantitative metrics for each baseline alongside AlayaWorld, including visual quality (FID/FVD), camera-following error, and revisit consistency scores. Where a baseline's public implementation does not support a particular input modality or resolution, we will state this explicitly and describe the closest matched setting used. We acknowledge that the current formulation is inadequate and will be fully revised. revision: yes

  3. Referee: §3.4: The runtime claim is central to the paper's positioning as 'real-time' and 'playable,' yet no latency measurements are provided. The paper states each chunk corresponds to roughly one second of video with four denoising steps but does not report wall-clock generation time, end-to-end interaction latency, or throughput.

    Authors: The referee is correct. The real-time claim is central to our positioning and is currently unsupported by quantitative evidence. We will add a runtime evaluation subsection reporting: (1) Wall-clock generation time per chunk (time from denoising start to frame output), measured on a specified GPU platform. (2) End-to-end interaction latency (time from user input to corresponding visual output appearing), which includes prompt switching overhead and cache update time. (3) Throughput in frames per second. (4) A breakdown of latency contributions: denoising steps, 3D cache rendering, history compression, and prompt switching. We will also clarify the hardware configuration used for all measurements. Without these numbers, the 'real-time' and 'playable' claims are indeed unsupported, and we will ensure the revised manuscript provides the necessary evidence. revision: yes

Circularity Check

0 steps flagged

No circularity: systems paper with no derivations, no fitted-parameter predictions, and no self-citation chain.

full rationale

AlayaWorld is a framework/systems paper that integrates known techniques (3D cache from GEN3C, history compression from Frame Preservation, error-bank training from Helios/Stable Video Infinity, DMD distillation, prompt switching) into a unified interactive world-generation pipeline. There are no mathematical derivations, no fitted constants presented as predictions, no uniqueness theorems invoked, and no equations that reduce to inputs by construction. The cited prior works (GEN3C, Frame Preservation, Helios, etc.) are by external author groups, not self-citations. While some authors (e.g., K. Zhang) appear on related prior work (Yume, Endless World), those works are cited only as related-work context, not as load-bearing premises for a derivation chain. The paper's claims are empirical system capabilities evaluated qualitatively in §4, with promised future quantitative evaluation. The absence of quantitative results is a correctness/evidence concern, not a circularity concern: no claim is shown to reduce to its own inputs by definition or by fit. The central content—combining spatial and temporal memory, error-bank perturbation of both memory and target, and chunk-level prompt switching—is described as engineering choices, not as derivations from prior results. No circularity is present.

Axiom & Free-Parameter Ledger

2 free parameters · 3 axioms · 1 invented entities

The ledger reflects the paper's nature as a systems integration effort. The free parameters are design choices, and the axioms are domain assumptions about the effectiveness of the integrated components. The 'error bank' is the most specific invented entity, but its independent evidence is limited to qualitative results.

free parameters (2)
  • Chunk size = 1 second of video (24fps)
    Stated in §4: 'each chunk corresponds to roughly one second of video'. This is a design choice affecting latency and stability.
  • Denoising steps = 4
    Stated in §4: 'each chunk is produced with four denoising steps'. A hyperparameter for the DMD-based distillation.
axioms (3)
  • domain assumption A 3D cache built from depth-unprojected frames provides sufficient spatial grounding for loop closure in revisited regions.
    Stated in §3.2 'Our approach': the cache 'supplies spatial persistence'. This assumes the cache captures enough geometric and appearance detail, which is not quantitatively verified.
  • domain assumption Injecting error-bank samples into both memory condition and target segment teaches the model to stabilize generation under corrupted history.
    Stated in §3.3 'Our approach'. The paper assumes this joint perturbation strategy is effective without ablation against single-side injection.
  • domain assumption LTX-2.3 is a suitable backbone for fine-tuning into an interactive world model.
    Stated in §3 and §4: 'fine-tuned from LTX-2.3'. Assumes the base model's latent space and architecture are compatible with the added modules.
invented entities (1)
  • Error bank no independent evidence
    purpose: Stores residual artifacts accumulated during rollout to be reused as structured perturbations during training.
    Introduced in §3.3. While conceptually similar to prior work [34], its specific implementation and effectiveness in this framework are not validated with quantitative metrics or ablation studies in the paper.

pith-pipeline@v1.1.0-glm · 18710 in / 2285 out tokens · 369818 ms · 2026-07-08T10:42:38.550444+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

99 extracted references · 99 canonical work pages · 31 internal anchors

  1. [1]

    Cosmos World Foundation Model Platform for Physical AI

    N. Agarwal, A. Ali, M. Bala, Y. Balaji, E. Barker, T. Cai, P . Chattopadhyay, Y. Chen, Y. Cui, Y. Ding, et al. Cosmos world foundation model platform for physical ai.arXiv preprint arXiv:2501.03575, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  2. [2]

    Happy oyster.https://happyoyster.cn/, 2026

    Alibaba ATH. Happy oyster.https://happyoyster.cn/, 2026

    work page 2026

  3. [3]

    Alonso, A

    E. Alonso, A. Jelley, V . Micheli, A. Kanervisto, A. Storkey, T. Pearce, and F. Fleuret. Diffusion for world modeling: Visual details matter in atari.Advances in Neural Information Processing Systems (NeurIPS), 37:58757–58791, 2024

    work page 2024

  4. [4]

    J. Bai, M. Xia, X. Fu, X. Wang, L. Mu, J. Cao, Z. Liu, H. Hu, X. Bai, P . Wan, et al. Recammaster: Camera-controlled generative rendering from a single video. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 14834–14844, 2025

    work page 2025

  5. [5]

    P . Ball, J. Bauer, et al. Genie 3: A new frontier for world models.https://deepmind.google/discover/ blog/genie-3-a-new-frontier-for-world-models/, 2025

    work page 2025

  6. [6]

    Goodbye Drift: Anchored Tree Sampling for Long-Horizon Video-to-Video Generation

    M. Bendel, S. W. Bailey, M. Vaidya, S. Badam, and X. He. Goodbye drift: Anchored tree sampling for long-horizon video-to-video generation.arXiv preprint arXiv:2605.20476, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  7. [7]

    Brooks, B

    T. Brooks, B. Peebles, C. Holmes, W. DePue, Y. Guo, L. Jing, D. Schnurr, J. Taylor, T. Luhman, E. Luhman, et al. Video generation models as world simulators.OpenAI Blog, 2024

    work page 2024

  8. [8]

    Bruce, M

    J. Bruce, M. D. Dennis, A. Edwards, J. Parker-Holder, Y. Shi, E. Hughes, M. Lai, A. Mavalankar, R. Steigerwald, C. Apps, et al. Genie: Generative interactive environments. InInternational Conference on Machine Learning (ICML), 2024. 10

    work page 2024

  9. [9]

    H. Che, X. He, Q. Liu, C. Jin, and H. Chen. Gamegen-x: Interactive open-world game video generation. InInternational Conference on Learning Representations (ICLR), pages 37546–37593, 2025

    work page 2025

  10. [10]

    S. Chen, C. Wei, S. Sun, P . Nie, K. Zhou, G. Zhang, M.-H. Yang, and W. Chen. Context forcing: Consistent autoregressive video generation with long context.arXiv preprint arXiv:2602.06028, 2026

    work page arXiv 2026

  11. [11]

    T. Chen, Z. Ding, A. Li, C. Zhang, Z. Xiao, Y. Wang, and C. Jin. Recurrent autoregressive diffusion: Global memory meets local attention.arXiv preprint arXiv:2511.12940, 2025

    work page arXiv 2025

  12. [12]

    Decart, Q

    E. Decart, Q. McIntyre, S. Campbell, X. Chen, and R. Wachen. Oasis: A universe in a transformer. https://oasis-model.github.io, 2024

    work page 2024

  13. [13]

    Joyai-echo: Pushing the frontier of long video generation

    Echo Team @ Joy Future Academy, JD. Joyai-echo: Pushing the frontier of long video generation. Technical report, Joy Future Academy, JD, May 2026

    work page 2026

  14. [14]

    R. Feng, H. Zhang, Z. Shu, Z. Yang, L. Tang, Z. Wang, A. Zheng, J. Xiao, Z. Liu, R. Chu, et al. The matrix: Infinite-horizon world generation with real-time moving control.Advances in Neural Information Processing Systems (NeurIPS), 38:87318–87344, 2026

    work page 2026

  15. [15]

    Veo.https://deepmind.google/models/veo/, 2024

    Google DeepMind. Veo.https://deepmind.google/models/veo/, 2024

    work page 2024

  16. [16]

    J. Gu, S. Wang, H. Zhao, T. Lu, X. Zhang, Z. Wu, S. Xu, W. Zhang, Y.-G. Jiang, and H. Xu. Reuse and diffuse: Iterative denoising for text-to-video generation.arXiv preprint arXiv:2309.03549, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  17. [17]

    J. Guo, Y. Ye, T. He, H. Wu, Y. Jiang, T. Pearce, and J. Bian. Mineworld: a real-time and open-source interactive world model on minecraft.arXiv preprint arXiv:2504.08388, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  18. [18]

    LTX-Video: Realtime Video Latent Diffusion

    Y. HaCohen, N. Chiprut, B. Brazowski, D. Shalem, D. Moshe, E. Richardson, E. Levin, G. Shiran, N. Zabari, O. Gordon, et al. Ltx-video: Realtime video latent diffusion.arXiv preprint arXiv:2501.00103, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  19. [19]

    H. He, Y. Xu, Y. Guo, G. Wetzstein, B. Dai, H. Li, and C. Yang. Cameractrl: Enabling camera control for text-to-video generation.arXiv preprint arXiv:2404.02101, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  20. [20]

    H. He, C. Yang, S. Lin, Y. Xu, M. Wei, L. Gui, Q. Zhao, G. Wetzstein, L. Jiang, and H. Li. Cameractrl ii: Dynamic scene exploration via camera-controlled video diffusion models. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 13416–13426, 2025

    work page 2025

  21. [21]

    X. He, C. Peng, Z. Liu, B. Wang, Y. Zhang, Q. Cui, F. Kang, B. Jiang, M. An, Y. Ren, et al. Matrix-game 2.0: An open-source real-time and streaming interactive world model.arXiv preprint arXiv:2508.13009, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  22. [22]

    Henschel, L

    R. Henschel, L. Khachatryan, H. Poghosyan, D. Hayrapetyan, V . Tadevosyan, Z. Wang, S. Navasardyan, and H. Shi. Streamingt2v: Consistent, dynamic, and extendable long video generation from text. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2568–2577, 2025

    work page 2025

  23. [23]

    Prompt-to-Prompt Image Editing with Cross Attention Control

    A. Hertz, R. Mokady, J. Tenenbaum, K. Aberman, Y. Pritch, and D. Cohen-Or. Prompt-to-prompt image editing with cross attention control.arXiv preprint arXiv:2208.01626, 2022

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  24. [24]

    Training-free Camera Control for Video Generation

    C. Hou and Z. Chen. Training-free camera control for video generation.arXiv preprint arXiv:2406.10126, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  25. [25]

    Huang, Z

    X. Huang, Z. Li, G. He, M. Zhou, and E. Shechtman. Self forcing: Bridging the train-test gap in autoregressive video diffusion.Advances in Neural Information Processing Systems (NeurIPS), 38:167283– 167308, 2026

    work page 2026

  26. [26]

    Kanervisto, D

    A. Kanervisto, D. Bignell, L. Y. Wen, M. Grayson, R. Georgescu, S. Valcarcel Macua, S. Z. Tan, T. Rashid, T. Pearce, Y. Cao, et al. World and human action models towards gameplay ideation.Nature, 638(8051):656–663, 2025

    work page 2025

  27. [27]

    J. Kim, J. Kang, J. Choi, and B. Han. Fifo-diffusion: Generating infinite videos from text without training.Advances in Neural Information Processing Systems (NeurIPS), 37:89834–89868, 2024. 11

    work page 2024

  28. [28]

    Kodaira, T

    A. Kodaira, T. Hou, J. Hou, M. Georgopoulos, F. Juefei-Xu, M. Tomizuka, and Y. Zhao. Streamdit: Real-time streaming text-to-video generation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 29200–29210, 2026

    work page 2026

  29. [29]

    W. Kong, Q. Tian, Z. Zhang, R. Min, Z. Dai, J. Zhou, J. Xiong, X. Li, B. Wu, J. Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models.arXiv preprint arXiv:2412.03603, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  30. [30]

    Kling.https://klingai.com/, 2024

    Kuaishou. Kling.https://klingai.com/, 2024

    work page 2024

  31. [31]

    J. Li, W. Feng, T.-J. Fu, X. Wang, S. Basu, W. Chen, and W. Y. Wang. T2v-turbo: Breaking the quality bottleneck of video consistency model with mixed reward feedback.Advances in Neural Information Processing Systems (NeurIPS), 37:75692–75726, 2024

    work page 2024

  32. [32]

    J. Li, J. Tang, Z. Xu, L. Wu, Y. Zhou, S. Shao, T. Yu, Z. Cao, and Q. Lu. Hunyuan-gamecraft: High-dynamic interactive game video generation with hybrid history condition.arXiv preprint arXiv:2506.17201, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  33. [33]

    R. Li, B. Yi, J. Liu, H. Gao, Y. Ma, and A. Kanazawa. Cameras as relative positional encoding.Advances in Neural Information Processing Systems (NeurIPS), 38:15984–16009, 2026

    work page 2026

  34. [34]

    W. Li, W. Pan, P .-C. Luan, Y. Gao, and A. Alahi. Stable video infinity: Infinite-length video generation with error recycling.arXiv preprint arXiv:2510.09212, 2025

    work page arXiv 2025

  35. [35]

    B. Lin, Y. Ge, X. Cheng, Z. Li, B. Zhu, S. Wang, X. He, Y. Ye, S. Yuan, L. Chen, et al. Open-sora plan: Open-source large video generation model.arXiv preprint arXiv:2412.00131, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  36. [36]

    H. Liu, W. Yan, M. Zaharia, and P . Abbeel. World model on million-length video and language with blockwise ringattention. InInternational Conference on Learning Representations (ICLR), pages 45953–45977, 2025

    work page 2025

  37. [37]

    K. Liu, W. Hu, J. Xu, Y. Shan, and S. Lu. Rolling forcing: Autoregressive long video diffusion in real time.arXiv preprint arXiv:2509.25161, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  38. [38]

    S. Liu, Y. Zhang, W. Li, Z. Lin, and J. Jia. Video-p2p: Video editing with cross-attention control. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8599–8608, 2024

    work page 2024

  39. [39]

    Z. Liu, X. Deng, S. Chen, A. Wang, Q. Guo, M. Han, Z. Xue, M. Chen, P . Luo, and L. Yang. Worldweaver: Generating long-horizon video worlds via rich perception.Advances in Neural Information Processing Systems (NeurIPS), 38:46268–46293, 2026

    work page 2026

  40. [40]

    S. Luo, Y. Tan, L. Huang, J. Li, and H. Zhao. Latent consistency models: Synthesizing high-resolution images with few-step inference.arXiv preprint arXiv:2310.04378, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  41. [41]

    Y. Luo, X. Shi, J. Bai, M. Xia, T. Xue, X. Wang, P . Wan, D. Zhang, and K. Gai. Camclonemaster: Enabling reference-based camera control for video generation. InSIGGRAPH Asia 2025 Conference Papers, pages 1–10, 2025

    work page 2025

  42. [42]

    Z. Lyu, C. Si, J. Song, Z. Yang, Y. Qiao, Z. Liu, and K.-Y. K. Wong. Fastercache: Training-free video diffusion model acceleration with high quality. InInternational Conference on Learning Representations (ICLR), pages 33132–33156, 2025

    work page 2025

  43. [43]

    G. Ma, H. Huang, K. Yan, L. Chen, N. Duan, S. Yin, C. Wan, R. Ming, X. Song, X. Chen, et al. Step- video-t2v technical report: The practice, challenges, and future of video foundation model.arXiv preprint arXiv:2502.10248, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  44. [44]

    X. Mao, Z. Li, C. Li, X. Xu, K. Ying, and K. Zhang. Yume1.5: A text-controlled interactive world generation model. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7752–7761, 2026

    work page 2026

  45. [45]

    X. Mao, S. Lin, Z. Li, C. Li, W. Peng, T. He, J. Pang, M. Chi, Y. Qiao, and K. Zhang. Yume: An interactive world generation model.arXiv preprint arXiv:2507.17744, 2025. 12

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  46. [46]

    Menapace, A

    W. Menapace, A. Siarohin, S. Lathuilière, P . Achlioptas, V . Golyanik, S. Tulyakov, and E. Ricci. Prompt- able game models: Text-guided game simulation via masked diffusion models.ACM Transactions on Graphics, 43(2):1–16, 2024

    work page 2024

  47. [47]

    Whamm! real-time world modelling of interac- tive environments

    Microsoft Research. Whamm! real-time world modelling of interac- tive environments. https://www.microsoft.com/en-us/research/articles/ whamm-real-time-world-modelling-of-interactive-environments/, 2025

    work page 2025

  48. [48]

    Parker-Holder, P

    J. Parker-Holder, P . Ball, J. Bruce, V . Dasagi, K. Holsheimer, C. Kaplanis, A. Moufarek, G. Scully, J. Shar, J. Shi, et al. Genie 2: A large-scale foundation world model. https://deepmind.google/discover/ blog/genie-2-a-large-scale-foundation-world-model, 2024

    work page 2024

  49. [49]

    Peebles and S

    W. Peebles and S. Xie. Scalable diffusion models with transformers. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 4195–4205, 2023

    work page 2023

  50. [50]

    R. Po, Y. Nitzan, R. Zhang, B. Chen, T. Dao, E. Shechtman, G. Wetzstein, and X. Huang. Long-context state-space video world models. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 8733–8744, 2025

    work page 2025

  51. [51]

    Movie Gen: A Cast of Media Foundation Models

    A. Polyak, A. Zohar, A. Brown, A. Tjandra, A. Sinha, A. Lee, A. Vyas, B. Shi, C.-Y. Ma, C.-Y. Chuang, et al. Movie gen: A cast of media foundation models.arXiv preprint arXiv:2410.13720, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  52. [52]

    X. Ren, T. Shen, J. Huang, H. Ling, Y. Lu, M. Nimier-David, T. Müller, A. Keller, S. Fidler, and J. Gao. Gen3c: 3d-informed world-consistent video generation with precise camera control. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6121–6132, 2025

    work page 2025

  53. [53]

    Rombach, A

    R. Rombach, A. Blattmann, D. Lorenz, P . Esser, and B. Ommer. High-resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10684–10695, 2022

    work page 2022

  54. [54]

    Introducing gen-3 alpha

    Runway. Introducing gen-3 alpha. https://runwayml.com/research/introducing-gen-3-alpha , 2024

    work page 2024

  55. [55]

    Progressive Distillation for Fast Sampling of Diffusion Models

    T. Salimans and J. Ho. Progressive distillation for fast sampling of diffusion models.arXiv preprint arXiv:2202.00512, 2022

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  56. [56]

    T. Shen, S. Bahmani, K. He, S. G. Srinivasan, T. Cao, J. Ren, R. Li, Z. Wang, N. Sharp, Z. Gojcic, et al. Lyra 2.0: Explorable generative 3d worlds.arXiv preprint arXiv:2604.13036, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  57. [57]

    J. Shin, Z. Li, R. Zhang, J.-Y. Zhu, J. Park, E. Shechtman, and X. Huang. Motionstream: Real-time video generation with interactive motion controls.arXiv preprint arXiv:2511.01266, 2025

    work page arXiv 2025

  58. [58]

    Y. Song, P . Dhariwal, M. Chen, and I. Sutskever. Consistency models. InInternational Conference on Machine Learning (ICML), 2023

    work page 2023

  59. [59]

    J. Tang, J. Liu, J. Li, L. Wu, H. Yang, P . Zhao, S. Gong, X. Yuan, S. Shao, L. Zhang, et al. Hunyuan- gamecraft-2: Instruction-following interactive game world model.arXiv preprint arXiv:2511.23429, 2025

    work page arXiv 2025

  60. [60]

    R. Team, Z. Gao, Q. Wang, Y. Zeng, J. Zhu, K. L. Cheng, Y. Li, H. Wang, Y. Xu, S. Ma, et al. Advancing open-source world models.arXiv preprint arXiv:2601.20540, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  61. [61]

    Hy-world 1.5: A systematic framework for interactive world modeling with real-time latency and geometric consistency.arXiv preprint, 2025

    Team HunyuanWorld. Hy-world 1.5: A systematic framework for interactive world modeling with real-time latency and geometric consistency.arXiv preprint, 2025

    work page 2025

  62. [62]

    Hy-world 2.0: A multi-modal world model for reconstructing, generating, and simulating 3d worlds.arXiv e-prints, 2026

    Team HY-World. Hy-world 2.0: A multi-modal world model for reconstructing, generating, and simulating 3d worlds.arXiv e-prints, 2026

    work page 2026

  63. [63]

    Hunyuanvideo 1.5 technical report, 2025

    Tencent Hunyuan Foundation Model Team. Hunyuanvideo 1.5 technical report, 2025

    work page 2025

  64. [64]

    H. Teng, H. Jia, L. Sun, L. Li, M. Li, M. Tang, S. Han, T. Zhang, W. Zhang, W. Luo, et al. Magi-1: Autoregressive video generation at scale.arXiv preprint arXiv:2505.13211, 2025. 13

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  65. [65]

    Valevski, Y

    D. Valevski, Y. Leviathan, M. Arar, and S. Fruchter. Diffusion models are real-time game engines. In International Conference on Learning Representations (ICLR), pages 73754–73776, 2025

    work page 2025

  66. [66]

    T. Wan, A. Wang, B. Ai, B. Wen, C. Mao, C.-W. Xie, D. Chen, F. Yu, H. Zhao, J. Yang, et al. Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  67. [67]

    F.-Y. Wang, Z. Huang, X. Shi, W. Bian, G. Song, Y. Liu, and H. Li. Animatelcm: Accelerating the animation of personalized diffusion models and adapters with decoupled consistency learning. In SIGGRAPH Asia 2024 Technical Communications, 2024

    work page 2024

  68. [68]

    J. Wang, L. Ye, T. Lu, J. Xiao, J. Zhang, Y. Guo, X. Liu, R. Chellappa, C. Peng, A. Yuille, et al. Evoworld: Evolving panoramic world generation with explicit 3d memory.arXiv preprint arXiv:2510.01183, 2025

    work page arXiv 2025

  69. [69]

    X. Wang, S. Zhang, H. Zhang, Y. Liu, Y. Zhang, C. Gao, and N. Sang. Videolcm: Video latent consistency model.arXiv preprint arXiv:2312.09109, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  70. [70]

    Y. Wang, Q. Zhang, S. Cai, T. Wu, J. Ackermann, Z. Kuang, Y. Zheng, F. Rajiˇ c, S. Tang, and G. Wetzstein. Bullettime: Decoupled control of time and camera pose for video generation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 18319–18330, 2026

    work page 2026

  71. [71]

    Z. Wang, Z. Liu, J. Li, K. Huang, B. Xu, F. Kang, M. An, P . Wang, B. Jiang, Y. Wei, et al. Matrix-game 3.0: Real-time and streaming interactive world model with long-horizon memory.arXiv preprint arXiv:2604.08995, 2026

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  72. [72]

    Z. Wang, Z. Yuan, X. Wang, Y. Li, T. Chen, M. Xia, P . Luo, and Y. Shan. Motionctrl: A unified and flexible motion controller for video generation. InACM SIGGRAPH 2024 Conference Papers, pages 1–11, 2024

    work page 2024

  73. [73]

    R. Wu, X. He, M. Cheng, T. Yang, Y. Zhang, Z. Kang, X. Cai, X. Wei, C. Guo, C. Li, et al. Infinite-world: Scaling interactive world models to 1000-frame horizons via pose-free hierarchical memory.arXiv preprint arXiv:2602.02393, 2026

    work page arXiv 2026

  74. [74]

    T. Wu, S. Yang, R. Po, Y. Xu, Z. Liu, D. Lin, and G. Wetzstein. Video world models with long-term spatial memory.Advances in Neural Information Processing Systems (NeurIPS), 38:49371–49393, 2026

    work page 2026

  75. [75]

    Z. Xiao, Y. Lan, Y. Zhou, W. Ouyang, S. Yang, Y. Zeng, and X. Pan. Worldmem: Long-term consistent world simulation with memory.Advances in Neural Information Processing Systems (NeurIPS), 38:49632– 49652, 2026

    work page 2026

  76. [76]

    D. Xu, W. Nie, C. Liu, S. Liu, J. Kautz, Z. Wang, and A. Vahdat. Camco: Camera-controllable 3d-consistent image-to-video generation.arXiv preprint arXiv:2406.02509, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  77. [77]

    M. Yang, J. Li, Z. Fang, S. Chen, Y. Yu, Q. Fu, W. Yang, and D. Ye. Playable game generation.arXiv preprint arXiv:2412.00887, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  78. [78]

    S. Yang, W. Huang, R. Chu, Y. Xiao, Y. Zhao, X. Wang, M. Li, E. Xie, Y. Chen, Y. Lu, et al. Longlive: Real-time interactive long video generation.arXiv preprint arXiv:2509.22622, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  79. [79]

    Z. Yang, J. Teng, W. Zheng, M. Ding, S. Huang, J. Xu, Y. Yang, W. Hong, X. Zhang, G. Feng, et al. Cogvideox: Text-to-video diffusion models with an expert transformer. InInternational Conference on Learning Representations (ICLR), pages 83048–83077, 2025

    work page 2025

  80. [80]

    T. Yin, M. Gharbi, T. Park, R. Zhang, E. Shechtman, F. Durand, and W. T. Freeman. Improved distribution matching distillation for fast image synthesis.Advances in Neural Information Processing Systems (NeurIPS), 37:47455–47487, 2024

    work page 2024

Showing first 80 references.