Pith. sign in

REVIEW 4 major objections 7 minor 1 cited by

Reviewed by Pith at T0; open to challenge.

T0 means a machine referee read the full paper against a public rubric. The mark states how deep the mechanical check went, never who wrote it. the ladder, T0–T4 →

T0 review · glm-5.2

Interactive world model sustains hour-long real-time video generation

2026-07-09 07:39 UTC pith:CQZV3HO7

load-bearing objection Open-source interactive world model with hour-long rollout claims, but stability evidence is qualitative only and self-contradicted by the paper's own limitations section. the 4 major comments →

arxiv 2607.07534 v1 pith:CQZV3HO7 submitted 2026-07-08 cs.CV

Infinite Worlds with Versatile Interactions

classification cs.CV
keywords interactive world modelautoregressive video generationdrift suppressiondistribution matching distillationcausal pretrainingreal-time video generationagentic harnessdiffusion transformer
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper claims that an autoregressive video diffusion model, trained with a causal pretraining paradigm and distilled into a few-step real-time generator, can sustain drift-free interactive world generation at 720p/60fps for over an hour without visible quality decay. The central mechanism is a two-stage pipeline: a base model trained with a Mixture of Bidirectional and Autoregressive attention mask to resist error accumulation, and a distilled student optimized via distribution matching distillation on its own self-rollout trajectories to suppress compounding drift. Around this generative core, the authors wrap an agentic harness where a vision-language model (the Director) reasons about scene state and proposes events, while the video generator (the Pilot) renders the resulting physical dynamics. The paper positions this combination as the bridge between short-lived video generation demos and genuinely open-ended, explorable worlds.

Core claim

The paper's central claim is that long-horizon drift in autoregressive video world models can be suppressed to the point of practical irrelevance by combining (1) a hybrid attention mask (MoBA) that regularizes pure teacher forcing during pretraining, preventing the model from passively copying context rather than predicting future frames, with (2) distribution matching distillation applied over the model's own long self-rollout trajectories, so the student is optimized on the distribution of states it will actually encounter at deployment time rather than only on teacher-forced states. The authors present a single 60-minute uninterrupted rollout across 20 scenarios as evidence that this is,

What carries the argument

Mixture of Bidirectional and Autoregressive (MoBA) attention mask, Distribution Matching Distillation (DMD) on self-rollout trajectories, Director-Pilot agentic harness

Load-bearing premise

The hour-long stability claim rests on a single qualitative rollout with no quantitative drift metric, no multiple seeds, and no automated quality assessment across the 60-minute timeline, so the assertion that stability is structural rather than clip-specific is not rigorously established.

What would settle it

Run multiple independent hour-long rollouts across different initial scenes and action sequences, measuring per-frame quality metrics (FID, LPIPS, temporal consistency) across the full timeline. If quality degrades measurably in any run, the structural-stability claim is weakened.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the anti-drift mechanism generalizes beyond the demonstrated rollout, interactive video world models could become practical backends for open-ended game environments and embodied simulation, replacing hand-authored content pipelines with generative ones.
  • The Director-Pilot architecture suggests a separation of concerns — semantic reasoning in language models, physical rendering in diffusion models — that could become a standard design pattern for interactive AI systems where causal reasoning and pixel-level generation require different computational substrates.
  • The release of both a 14B base model and a 1.3B distilled variant on a single GPU lowers the deployment barrier enough that independent developers could build on real-time interactive world generation without proprietary infrastructure.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper attributes stability jointly to MoBA and to DMD-on-self-rollout, but does not isolate their contributions through ablation. If the self-rollout DMD is the dominant factor, the MoBA mask may be replaceable by simpler regularization, which would simplify future training pipelines.
  • The hour-long stability claim rests on a single qualitative rollout with no quantitative drift metric (FID, LPIPS, temporal consistency) measured across the timeline. A natural testable extension would be to run multiple seeds and scenarios with automated quality scoring to determine whether stability holds uniformly or is scenario-dependent.
  • The paper acknowledges that the model lacks genuine long-term memory — revisited regions are regenerated rather than recalled. This implies the world is persistent in appearance but not in identity, which would become visible in any task requiring the agent to reference a previously visited location by its specific features rather than its general category.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper presents LingBot-World 2.0 (LingBot-World-Infinity), an interactive causal video world model with four claimed advances: (1) hour-long stable generation via causal pretraining with a Mixture of Bidirectional and Autoregressive (MoBA) attention mask, (2) real-time 720p/60fps distilled model via consistency + distribution matching distillation, (3) a rich action space (combat, archery, spell-casting, shooting, environmental events), and (4) a Director-Pilot agentic harness where a VLM proposes events and a video generator renders them. The system is released with checkpoints and code. The central empirical claim is that the distilled model sustains drift-free interactive generation for over an hour, demonstrated via a single 60-minute rollout (Fig. 10) across 20 scenarios.

Significance. The paper tackles a genuine and important problem in interactive world modeling: long-horizon stability under autoregressive error accumulation. The combination of MoBA attention, DMD-on-self-rollout training, and the Director-Pilot harness is a reasonable engineering response. The release of checkpoints, code, and a real-time system at 720p/60fps is a concrete community contribution. However, the load-bearing claim of hour-long drift-free generation is supported only by qualitative evidence, and the paper's own Limitations section partially contradicts the headline. The MoBA and DMD contributions are not individually ablated, making causal attribution of the stability claim difficult.

major comments (4)
  1. Sec. 5.1, contributions bullet 2, and Fig. 10: The claim that the model is 'verified by over an hour of continuous generation without quality loss' and that 'visual quality shows no perceptible decay' is supported solely by a single qualitative rollout with selected frames at ~20 timestamps. There are no quantitative drift metrics (FID, LPIPS, temporal consistency, action-response accuracy), no multiple seeds, and no baseline comparison on the same long-horizon protocol. For a claim described as 'verified' and 'structural,' this evidence is insufficient. At minimum, the authors should compute frame-level or chunk-level quality metrics across the 60-minute timeline and report them, ideally across multiple sessions.
  2. Sec. 6 (Limitations) vs. Sec. 5.1 and contributions bullet 2: The Limitations section states 'specific characters can subtly change in appearance and the overall art style may gradually drift' over very long rollouts. This directly contradicts the headline claim of 'drift-free' generation and 'no perceptible decay.' The authors should either soften the headline claim to match the acknowledged limitations (e.g., 'low drift' with quantified boundaries) or remove the 'drift-free' language from the abstract, contributions, and Sec. 5.1.
  3. Sec. 3.2 (MoBA) and Sec. 3.3 (DMD-on-self-rollout): Both the MoBA attention mask and the DMD self-rollout training are motivated as anti-drift mechanisms, but no ablation isolates the contribution of either. The claim that stability is 'structural' and attributable to the causal pretraining paradigm cannot be verified without knowing whether MoBA, DMD-on-self-rollout, or their combination is responsible. An ablation table reporting drift metrics (even qualitative degradation timestamps) for: (a) teacher forcing only, (b) MoBA only, (c) DMD-on-teacher-forced states only, (d) both combined, would substantially strengthen the contribution.
  4. Table 1: The comparison table uses categorical labels ('Minutes' vs 'Hours (Infinite)') for generation duration without quantitative measurements. Claiming 'Hours (Infinite)' for the proposed model while labeling all baselines as 'Minutes' is not a fair comparison unless the baselines were tested under the same long-horizon protocol and failed. The table should either report measured maximum stable durations for all models or clearly state that baseline entries are based on published reports.
minor comments (7)
  1. Sec. 3.1: The Hume epigraph is decorative and does not contribute to the technical exposition. Consider removing.
  2. Sec. 3.2, Fig. 4: The MoBA mask is described verbally but the exact mask structure (which frames attend bidirectionally vs. autoregressively) is not fully specified. A formal matrix definition or clearer diagram annotation would aid reproducibility.
  3. Sec. 4.3.2: The dynamic KV-cache management mechanism is described qualitatively ('adapt the cache on the fly according to the current control signal and input state') but the scheduling policy, retention criteria, and cache size budget are not specified.
  4. Fig. 11 and Fig. 12: The qualitative comparisons show frames at 5s, 15s, 25s, 35s, 45s, 60s for baselines and the proposed model, but no metric is provided to quantify the visual differences. Even a simple FID or CLIP-score per timestamp would strengthen the comparison.
  5. Sec. 2: The data pipeline is described in detail but the total dataset size (number of videos, hours of footage) is not reported. This makes it difficult to assess the training data's adequacy.
  6. The paper uses both 'LingBot-World 2.0' and 'LingBot-World-Infinity' interchangeably. Consider standardizing on one name.
  7. Sec. 4.2: The Director-Pilot framework is described as a 'novel' contribution, but the relationship to prior LLM-directed video generation work (e.g., Free-Bloom [25], LLM-Grounded Video Diffusion [29]) should be discussed to position the contribution.

Simulated Author's Rebuttal

4 responses · 0 unresolved

We thank the referee for a careful and constructive review. The referee identifies four substantive issues: (1) insufficient quantitative evidence for the hour-long stability claim, (2) a contradiction between the headline 'drift-free' language and the Limitations section, (3) missing ablations for MoBA and DMD-on-self-rollout, and (4) unfair categorical labels in Table 1. We agree with the substance of all four points and will revise accordingly. Below we address each in turn.

read point-by-point responses
  1. Referee: Sec. 5.1, contributions bullet 2, and Fig. 10: The claim that the model is 'verified by over an hour of continuous generation without quality loss' and that 'visual quality shows no perceptible decay' is supported solely by a single qualitative rollout with selected frames at ~20 timestamps. There are no quantitative drift metrics, no multiple seeds, and no baseline comparison on the same long-horizon protocol.

    Authors: The referee is correct that the current evidence for the hour-long stability claim is purely qualitative. We will address this by computing frame-level and chunk-level quality metrics (FID, LPIPS, temporal consistency via optical-flow warping error, and action-response accuracy) across the full 60-minute timeline, sampled at regular intervals. We will additionally run at least three independent 60-minute sessions with different seeds and report the metrics across all sessions. We will present these as a quantitative drift curve alongside the existing qualitative figure. We acknowledge that the single-rollout evidence in the current manuscript is insufficient to support the word 'verified,' and we will adjust our language accordingly. revision: yes

  2. Referee: Sec. 6 (Limitations) vs. Sec. 5.1 and contributions bullet 2: The Limitations section states 'specific characters can subtly change in appearance and the overall art style may gradually drift' over very long rollouts. This directly contradicts the headline claim of 'drift-free' generation and 'no perceptible decay.'

    Authors: The referee has identified a genuine internal contradiction in the manuscript. The Limitations section acknowledges that subtle character appearance changes and gradual art-style drift can occur over very long rollouts, while the abstract, contributions, and Sec. 5.1 use the language 'drift-free' and 'no perceptible decay.' These two statements cannot both be true. We will resolve this by removing the absolute term 'drift-free' from the abstract, contributions list, and Sec. 5.1, replacing it with a more precise characterization such as 'low drift' or 'substantially reduced drift,' accompanied by the quantitative boundaries from the new drift metrics described in our response to Comment 1. The Limitations text will be retained and, where appropriate, sharpened to reference the measured drift rates so that the headline claim and the acknowledged limitations are consistent. revision: yes

  3. Referee: Sec. 3.2 (MoBA) and Sec. 3.3 (DMD-on-self-rollout): Both the MoBA attention mask and the DMD self-rollout training are motivated as anti-drift mechanisms, but no ablation isolates the contribution of either. The claim that stability is 'structural' and attributable to the causal pretraining paradigm cannot be verified without knowing whether MoBA, DMD-on-self-rollout, or their combination is responsible.

    Authors: We agree that the current manuscript does not provide the ablations needed to attribute the stability improvement to specific components. We will conduct and report a 2x2 ablation along the lines the referee suggests: (a) teacher forcing only (no MoBA, no DMD-on-self-rollout), (b) MoBA only, (c) DMD-on-self-rollout applied to a teacher-forced backbone (no MoBA), and (d) both combined. For each configuration, we will report the same drift metrics (FID, LPIPS, temporal consistency) measured over a fixed long-horizon protocol (e.g., 30 minutes), as well as qualitative degradation timestamps. This will allow causal attribution of the stability properties to MoBA, DMD-on-self-rollout, or their interaction. We will also soften the claim that stability is 'structural' until the ablation results justify the specific attribution. revision: yes

  4. Referee: Table 1: The comparison table uses categorical labels ('Minutes' vs 'Hours (Infinite)') for generation duration without quantitative measurements. Claiming 'Hours (Infinite)' for the proposed model while labeling all baselines as 'Minutes' is not a fair comparison unless the baselines were tested under the same long-horizon protocol and failed.

    Authors: The referee is right that the categorical labels in Table 1 are not backed by a controlled comparison. We will revise Table 1 in one of two ways, depending on feasibility: (Option A) If we can obtain and run the open-source baselines (M-G 3.0, D-W, LingBot-World 1.0, SANA-WM) under the same long-horizon protocol, we will report measured maximum stable durations for all models. (Option B) If running all baselines for extended periods is not feasible due to compute or access constraints, we will clearly annotate each baseline entry with its source (e.g., 'per published report' or 'our measurement') and replace the categorical labels with the specific durations cited. In either case, we will remove the label 'Hours (Infinite)' for our own model and replace it with the measured stable duration from our quantitative evaluation, and we will add a footnote clarifying the protocol under which each duration was determined. revision: yes

Circularity Check

0 steps flagged

No significant circularity: the paper's claims are empirical, not derived from self-cited premises by construction.

full rationale

The paper describes an interactive world model system (LingBot-World-Infinity) with four claimed upgrades: long-horizon stability, real-time distillation, diverse interactions, and an agentic harness. The central claim—hour-long stability—is an empirical assertion verified (or under-verified, per the skeptic) by a single 60-minute rollout (Fig. 10), not a quantity derived from equations or prior cited results. The technical derivations (Eqs. 1–4) are standard formulations: Eq. 1 is the causal factorization of a video sequence, Eqs. 2–4 are flow-matching, consistency distillation, and DMD objectives drawn from external methods (rectified flow, consistency models [39], DMD [58]). None of these equations reduce to their own inputs by construction. The paper does cite its predecessor LingBot-World 1.0 [44] and related work by overlapping authors (CausalCine [32], WorldDirector [46]), but these citations provide context or architectural choices, not load-bearing premises from which the paper's results are derived. The MoBA attention mask (Sec. 3.2) is a proposed architectural component, not a result claimed to follow from a self-cited theorem. The distillation pipeline (Sec. 3.3) combines externally established techniques (consistency distillation, DMD, self-forcing [26]) without circular dependency. The under-verification of the stability claim (single qualitative rollout, no quantitative drift metrics, Limitations section acknowledging drift) is a correctness and evidence concern, not a circularity issue—the claim is not constructed to be true by definition. No step in the paper's chain reduces to its inputs by construction.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 2 invented entities

The paper introduces two novel architectural entities (MoBA mask, Director-Pilot harness) but provides no ablation for either. The free parameters are numerous and unstated, reflecting the paper's nature as a systems/demo paper rather than a controlled scientific study. The axioms are mostly standard domain assumptions from the video generation literature, with two exceptions (teacher forcing overfitting, DMD-on-rollout drift reduction) that are stated but not independently verified.

free parameters (5)
  • Model architecture hyperparameters (N DiT blocks, hidden dim, attention heads) = Not stated
    The 14B and 1.3B model architectures are not specified in the paper. These are design choices that affect all results.
  • Training data volume and composition = Not stated
    The data pipeline is described qualitatively but the total dataset size, source proportions, and number of training samples are not given.
  • DMD self-rollout trajectory length = Not stated
    Sec. 3.3 states DMD is applied 'over long self-rollout trajectories' but the length is unspecified. This is a key hyperparameter for the anti-drift claim.
  • KV-cache scheduling policy parameters = Not stated
    Sec. 4.3.2 describes dynamic KV-cache management but does not specify the retention/eviction criteria.
  • Consistency distillation step size (Δt) = Not stated
    Eq. 3 uses Δt > 0 but the value is not given.
axioms (5)
  • domain assumption Causal factorization of video generation (Eq. 1): each state depends solely on historical context and current input.
    Sec. 3.1. Standard autoregressive assumption; well-justified for sequential generation.
  • domain assumption Teacher forcing causes overfitting and quality degradation as context grows, motivating MoBA mask.
    Sec. 3.2. Stated as empirical observation but no ablation is provided to isolate this effect.
  • domain assumption DMD over self-rollout trajectories reduces accumulated drift.
    Sec. 3.3. Follows Self-Forcing [26] but the specific claim about drift reduction is not quantitatively verified.
  • domain assumption VLM-based Director can perform causal reasoning sufficient for world simulation.
    Sec. 4.2. The paper assumes a VLM (Qwen3.5/3.6) can reason about physical and semantic consequences. No evaluation of reasoning quality is provided.
  • standard math Rectified-flow interpolation is the correct training target for flow matching.
    Eq. 2. Standard from the flow matching literature.
invented entities (2)
  • MoBA (Mixture of Bidirectional and Autoregressive) Attention Mask no independent evidence
    purpose: Hybrid self-attention mask combining teacher forcing with bidirectional attention to prevent overfitting and enable flexible-length generation.
    Introduced in Sec. 3.2. No ablation isolating its effect from the DMD self-rollout training. The claim that it mitigates overfitting is stated but not independently verified.
  • Director-Pilot Co-Simulation Framework no independent evidence
    purpose: VLM (Director) proposes events; video generator (Pilot) renders them. Bridges semantic reasoning and physical simulation.
    Sec. 4.2. The framework is a system design choice. No evaluation of the Director's reasoning quality or the Pilot's faithfulness to Director proposals is provided.

pith-pipeline@v1.1.0-glm · 20626 in / 3542 out tokens · 437811 ms · 2026-07-09T07:39:27.100796+00:00 · methodology

0 comments
read the original abstract

We present LingBot-World 2.0 (also known as LingBot-World-Infinity), an advanced iteration of LingBot-World featuring four distinct upgrades. (1) Our model achieves an unbounded interaction horizon while maintaining consistent output quality, benefiting from a carefully crafted causal pretraining paradigm. (2) Through distilling a real-time variant from the base model, our system guarantees rapid response time, sufficient to drive 720p video streams at 60 fps. (3) Compared to the previous version, this update introduces highly diverse interactive elements, comprising a broader spectrum of actions (e.g., attacking, archery, spell-casting, and shooting) alongside a richer variety of text-driven events. (4) We pioneer the integration of an agentic harness within the domain of world modeling, wherein a pilot agent is tasked with planning and executing character behaviors, while a director agent is responsible for synthesizing novel environmental elements as the scene progresses. Additionally, to facilitate a shared experience, we develop an interface that permits multiple players to simultaneously immerse themselves in this vivid world simulator. We pair our primary 14B model with a lightweight 1.3B counterpart, which supports effortless deployment on a single GPU.

Figures

Figures reproduced from arXiv: 2607.07534 by Hanlin Wang, Haojie Zhang, Hao Ouyang, Jiahao Wang, Jian Gao, Jiapeng Zhu, Jingye Chen, Ka Leong Cheng, Qingyan Bai, Qiuyu Wang, Tianrui Feng, Xing Zhu, Yao Yao, Yichong Lu, Yinghao Xu, Yufeng Yuan, Yujun Shen, Yuzheng Liu, Zelin Gao, Zichen Liu.

Figure 1
Figure 1. Figure 1: LingBot-World-Infinity generates infinite worlds in real time, featuring versatile interactions. 1 arXiv:2607.07534v1 [cs.CV] 8 Jul 2026 [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the proposed data engine. Heterogeneous raw videos are temporally segmented, filtered, and routed to category-specific annotation pipelines, producing optimized chunk-wise captions. global context, then produces event-level annotations for each temporal chunk, and finally composes these annotations into visually grounded and temporally aligned captions. This hierarchical design is consistent wi… view at source ↗
Figure 3
Figure 3. Figure 3: Overview of LingBot-World-Infinity Pipeline. An interactive world simulator is implemented as a causal video model. Our Infinity World is initialized from an initial image and its background description. The future world states are then autoregressively generated, conditioned on the historical context and user inputs (camera poses and prompts). 3.1 Formulation “The cause must be prior to the effect.” — Dav… view at source ↗
Figure 4
Figure 4. Figure 4: Overview of LingBot-World-Infinity DiT Block and MoBA Attention Mask. The action comprises camera poses and chunk-wise prompts, injected into the DiT block to enable user interaction. For self-attention, a bidirectional block is appended to teacher forcing mask, enabling autoregressive generation while preserving visual fidelity. For cross-attention, the autoregressive component attends to a background pro… view at source ↗
Figure 5
Figure 5. Figure 5: Overview of the Agentic Interaction Harness. Users can either interact with the existing world through semantic or object-centric actions, or intervene by introducing high-level textual events. The VLM (Director) performs causal reasoning and proposes coherent event updates, while the Video Generator (Pilot) grounds these semantic decisions into physically consistent video rollouts, enabling continuous int… view at source ↗
Figure 6
Figure 6. Figure 6: In the tracking-mode interface, the Vision-Language Model (VLM) comprehends interactive objects within the scene, while the tracking model continuously tracks these targets to display dynamic interactive floating windows (event cards) in real-time. Powered by the "Director-Pilot" co-simulation framework, the model demonstrates robust interactive capabilities by performing causal reasoning based on user act… view at source ↗
Figure 7
Figure 7. Figure 7: Our world model enables controllable world exploration with versatile interactions. It supports flexible prompt switching across different world horizons, allowing the scene semantics to evolve smoothly, while also enabling controllable navigation of diverse protagonists and objects throughout the generated world (1/2). the same asynchronous streaming pipeline, improving perceptual quality while introducin… view at source ↗
Figure 8
Figure 8. Figure 8: Our world model enables controllable world exploration with versatile interactions. It supports flexible prompt switching across different world horizons, allowing the scene semantics to evolve smoothly, while also enabling controllable navigation of diverse protagonists and objects throughout the generated world (2/2). 4.3.2 Dynamic KV Cache Management A key component of our real-time system is a dynamic … view at source ↗
Figure 9
Figure 9. Figure 9: Interface of our interactive system. The center shows the live generation viewport; the bottom panel exposes the low-level control scheme (WASD for movement, IJKL for view control); and the right-hand panel is the agentic control surface, which lists the VLM-proposed “Event Proposals,” each bound to a hotkey. Fixed keys (Space, P) are always available, U/O and F/G carry context-aware character actions and … view at source ↗
Figure 10
Figure 10. Figure 10: Hour-long world rollout. We sample frames from a single 60-minute generated session, covering 20 distinct scenarios. The timeline shows that the model can maintain coherent scene structure, visual quality over an extended uninterrupted rollout, providing a qualitative stress test for long-horizon stability. examining the capabilities that emerge at long horizons. We then report results for the causal pret… view at source ↗
Figure 11
Figure 11. Figure 11: Qualitative comparisons. Our model maintains stable visual and physical consistency over long-horizon generation. shown in [PITH_FULL_IMAGE:figures/full_fig_p014_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Qualitative comparisons on causal pretraining. Our model shows stable performance compared with baselines. model from compounding its own errors. 6 Conclusion and Discussion Conclusion. We present LingBot-World-Infinity, an open causal video generation model for interactive world modeling. It pairs state-of-the-art visual quality with strong resistance to drift, and from it we distill a real-time model th… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Native Video-Action Pretraining for Generalizable Robot Control

    cs.RO 2026-07 conditional novelty 5.0

    A video-action foundation model pretrained natively for embodiment achieves few-shot generalization and 225 Hz real-time closed-loop robot control.

Reference graph

Works this paper leans on

64 extracted references · 64 canonical work pages · cited by 1 Pith paper · 25 internal anchors

  1. [1]

    Cosmos 3: Omnimodal World Models for Physical AI

    Niket Agarwal, Arslan Ali, Jon Allen, Martin Antolini, Adeline Aubame, Alisson Azzolini, Junjie Bai, Maciej Bala, Yogesh Balaji, Josh Bapst, et al. Cosmos 3: Omnimodal world models for physical ai.arXiv preprint arXiv:2606.02800, 2026

  2. [2]

    Egocentric-10k, 2025

    Build AI. Egocentric-10k, 2025

  3. [3]

    Philip J. Ball, Jakob Bauer, Frank Belletti, Bethanie Brownfield, Ariel Ephrat, Shlomi Fruchter, Agrim Gupta, Kristian Holsheimer, Aleksander Holynski, Jiri Hron, Christos Kaplanis, Marjorie Limont, Matt McGill, Yanko Oliveira, Jack Parker- Holder, Frank Perbet, Guy Scully, Jeremy Shar, Stephen Spencer, Omer Tov, Ruben Villegas, Emma Wang, Jessica Yung, C...

  4. [4]

    My view is the best view: Procedure learning from egocentric videos

    Siddhant Bansal, Chetan Arora, and CV Jawahar. My view is the best view: Procedure learning from egocentric videos. In European Conference on Computer Vision. Springer, 2022

  5. [5]

    Gamegen-x: Interactive open-world game video generation

    Haoxuan Che, Xuanhua He, Quande Liu, Cheng Jin, and Hao Chen. Gamegen-x: Interactive open-world game video generation. InInt. Conf. Learn. Represent., 2025

  6. [6]

    Memdreamer: Decoupling perception and reasoning for long video understanding via hierarchical graph memory and agentic retrieval mechanism, 2026

    Cong Chen, Guo Gan, Kaixiang Ji, ZhaoYang Zhang, Zhen Yang, Guangming Yao, Hao Chen, Jingdong Chen, Yi Yuan, and Chunhua Shen. Memdreamer: Decoupling perception and reasoning for long video understanding via hierarchical graph memory and agentic retrieval mechanism, 2026. 16

  7. [7]

    SkyReels-V2: Infinite-length Film Generative Model

    Guibin Chen, Dixuan Lin, Jiangping Yang, Chunze Lin, Junchen Zhu, Mingyuan Fan, Hao Zhang, Sheng Chen, Zheng Chen, Chengcheng Ma, et al. Skyreels-v2: Infinite-length film generative model.arXiv preprint arXiv:2504.13074, 2025

  8. [8]

    Panda-70m: Captioning 70m videos with multiple cross-modality teachers

    Tsai-Shien Chen, Aliaksandr Siarohin, Willi Menapace, Ekaterina Deyneka, Hsiang-wei Chao, Byung Eun Jeon, Yuwei Fang, Hsin-Ying Lee, Jian Ren, Ming-Hsuan Yang, et al. Panda-70m: Captioning 70m videos with multiple cross-modality teachers. InInt. Conf. Comput. Vis., 2024

  9. [9]

    LongLive-2.0: An NVFP4 Parallel Infrastructure for Long Video Generation

    Yukang Chen, Luozhou Wang, Wei Huang, Shuai Yang, Bohan Zhang, Yicheng Xiao, Ruihang Chu, Weian Mao, Qixin Hu, Shaoteng Liu, Yuyang Zhao, Huizi Mao, Ying-Cong Chen, Enze Xie, Xiaojuan Qi, and Song Han. Longlive2.0: An nvfp4 parallel infrastructure for long video generation.arXiv preprint arXiv: 2605.18739, 2026

  10. [10]

    Abot-physworld: Interactive world foundation model for robotic manipulation with physics alignment

    Yuzhi Chen, Ronghan Chen, Dongjie Huo, Yandan Yang, Dekang Qi, Haoyun Liu, Tong Lin, Shuang Zeng, Junjin Xiao, Xinyuan Chang, et al. Abot-physworld: Interactive world foundation model for robotic manipulation with physics alignment. arXiv preprint arXiv:2603.23376, 2026

  11. [11]

    PaddleOCR 3.0 Technical Report

    Cheng Cui, Ting Sun, Manhui Lin, Tingquan Gao, Yubo Zhang, Jiaxuan Liu, Xueqing Wang, Zelun Zhang, Changda Zhou, Hongen Liu, et al. Paddleocr 3.0 technical report.arXiv preprint arXiv:2507.05595, 2025

  12. [12]

    The epic-kitchens dataset: Collection, challenges and baselines.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020

    Dima Damen, Hazel Doughty, Giovanni Maria Farinella, Sanja Fidler, Antonino Furnari, Evangelos Kazakos, Davide Moltisanti, Jonathan Munro, Toby Perrett, Will Price, et al. The epic-kitchens dataset: Collection, challenges and baselines.IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020

  13. [13]

    xDiT: an Inference Engine for Diffusion Transformers (DiTs) with Massive Parallelism

    Jiarui Fang, Jinzhe Pan, Xibo Sun, Aoyu Li, and Jiannan Wang. xdit: an inference engine for diffusion transformers (dits) with massive parallelism.ArXiv, abs/2411.01738, 2024

  14. [14]

    Streamdiffusionv2: A streaming system for dynamic and interactive video generation

    Tianrui Feng, Zhi Li, Shuo Yang, Haocheng Xi, Muyang Li, Xiuyu Li, Lvmin Zhang, Keting Yang, Kelly Peng, Song Han, et al. Streamdiffusionv2: A streaming system for dynamic and interactive video generation. InMLSYS, 2026

  15. [15]

    Flashdreams: High-performance inference and serving for interactive autoregressive video and world models.https://github.com/NVIDIA/flashdreams, 2026

    FlashDreams Contributors. Flashdreams: High-performance inference and serving for interactive autoregressive video and world models.https://github.com/NVIDIA/flashdreams, 2026

  16. [16]

    Gigaworld-0: World models as data engine to empower embodied ai, 2025

    GigaAI. Gigaworld-0: World models as data engine to empower embodied ai, 2025

  17. [17]

    Kristen Grauman, Andrew Westbury, Eugene Byrne, Zachary Chavis, Antonino Furnari, Rohit Girdhar, Jackson Hamburger, Hao Jiang, Miao Liu, Xingyu Liu, Miguel Martin, Tushar Nagarajan, Ilija Radosavovic, Santhosh Kumar Ramakrishnan, Fiona Ryan, Jayant Sharma, Michael Wray, Mengmeng Xu, Eric Zhongcong Xu, Chen Zhao, Siddhant Bansal, Dhruv Batra, Vincent Carti...

  18. [18]

    LTX-Video: Realtime Video Latent Diffusion

    Yoav HaCohen, Nisan Chiprut, Benny Brazowski, Daniel Shalem, Dudu Moshe, Eitan Richardson, Eran Levin, Guy Shiran, Nir Zabari, Ori Gordon, Poriya Panet, Sapir Weissbuch, Victor Kulikov, Yaki Bitterman, Zeev Melumian, and Ofir Bibi. Ltx-video: Realtime video latent diffusion.arXiv preprint arXiv:2501.00103, 2024

  19. [19]

    Happy oyster: Real-time world model for interactive creation, 2026

    Happy Oyster. Happy oyster: Real-time world model for interactive creation, 2026

  20. [20]

    CameraCtrl: Enabling Camera Control for Text-to-Video Generation

    Hao He, Yinghao Xu, Yuwei Guo, Gordon Wetzstein, Bo Dai, Hongsheng Li, and Ceyuan Yang. Cameractrl: Enabling camera control for text-to-video generation.arXiv preprint arXiv:2404.02101, 2024

  21. [21]

    Cameractrl ii: Dynamic scene exploration via camera-controlled video diffusion models

    Hao He, Ceyuan Yang, Shanchuan Lin, Yinghao Xu, Meng Wei, Liangke Gui, Qi Zhao, Gordon Wetzstein, Lu Jiang, and Hongsheng Li. Cameractrl ii: Dynamic scene exploration via camera-controlled video diffusion models. InIEEE Conf. Comput. Vis. Pattern Recog., 2025

  22. [22]

    Matrix-game 2.0: An open-source real-time and streaming interactive world model

    Xianglong He, Chunli Peng, Zexiang Liu, Boyang Wang, Yifan Zhang, Qi Cui, Fei Kang, Biao Jiang, Mengyin An, Yangyang Ren, et al. Matrix-game 2.0: An open-source real-time and streaming interactive world model.arXiv preprint arXiv:2508.13009, 2025

  23. [23]

    Yoon, Mouli Sivapurapu, and Jian Zhang

    Ryan Hoque, Peide Huang, David J. Yoon, Mouli Sivapurapu, and Jian Zhang. Egodex: Learning dexterous manipulation from large-scale egocentric video, 2025. 17

  24. [24]

    StoryAgent: Customized Storytelling Video Generation via Multi-Agent Collaboration

    Panwen Hu, Jin Jiang, Jianqi Chen, Mingfei Han, Shengcai Liao, Xiaojun Chang, and Xiaodan Liang. Storyagent: Customized storytelling video generation via multi-agent collaboration.ArXiv, abs/2411.04925, 2024

  25. [25]

    Free-Bloom: Zero-Shot Text-to-Video Generator with LLM Director and LDM Animator

    Hanzhuo Huang, Yufan Feng, Cheng Shi, Lan Xu, Jingyi Yu, and Sibei Yang. Free-bloom: Zero-shot text-to-video generator with llm director and ldm animator.ArXiv, abs/2309.14494, 2023

  26. [26]

    Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion

    Xun Huang, Zhengqi Li, Guande He, Mingyuan Zhou, and Eli Shechtman. Self forcing: Bridging the train-test gap in autoregressive video diffusion.arXiv preprint arXiv:2506.08009, 2025

  27. [27]

    Oxford University Press, 2000

    David Hume.A treatise of human nature. Oxford University Press, 2000

  28. [28]

    Epic-tent: An egocentric video dataset for camping tent assembly

    Youngkyoon Jang, Brian Sullivan, Casimir Ludwig, Iain Gilchrist, Dima Damen, and Walterio Mayol-Cuevas. Epic-tent: An egocentric video dataset for camping tent assembly. InProceedings of the IEEE/CVF International Conference on Computer Vision Workshops, 2019

  29. [29]

    Llm-grounded video diffusion models

    Long Lian, Baifeng Shi, Adam Yala, Boyi Li, et al. Llm-grounded video diffusion models. InInt. Conf. Learn. Represent., volume 2024, 2024

  30. [30]

    Autoregressive adversarial post-training for real-time interactive video generation

    Shanchuan Lin, Ceyuan Yang, Hao He, Jianwen Jiang, Yuxi Ren, Xin Xia, Yang Zhao, Xuefeng Xiao, and Lu Jiang. Autoregressive adversarial post-training for real-time interactive video generation. InAdv. Neural Inform. Process. Syst., 2025

  31. [31]

    Xiaofeng Mao, Zhen Li, Chuanhao Li, Xiaojie Xu, Kaining Ying, and Kaipeng Zhang. Yume1. 5: A text-controlled interactive world generation model. InIEEE Conf. Comput. Vis. Pattern Recog., 2026

  32. [32]

    CausalCine: Real-Time Autoregressive Generation for Multi-Shot Video Narratives

    Yihao Meng, Zichen Liu, Hao Ouyang, Qiuyu Wang, Ka Leong Cheng, Yue Yu, Hanlin Wang, Haobo Li, Jiapeng Zhu, Yanhong Zeng, et al. Causalcine: Real-time autoregressive generation for multi-shot video narratives.arXiv preprint arXiv:2605.12496, 2026

  33. [33]

    Openvid-1m: A large-scale high-quality dataset for text-to-video generation

    Kepan Nan, Rui Xie, Penghao Zhou, Tiehan Fan, Zhenheng Yang, Zhijie Chen, Xiang Li, Jian Yang, and Ying Tai. Openvid-1m: A large-scale high-quality dataset for text-to-video generation. InInt. Conf. Learn. Represent., volume 2025, 2025

  34. [34]

    Qwen3.5: Towards native multimodal agents, February 2026

    Qwen Team. Qwen3.5: Towards native multimodal agents, February 2026

  35. [35]

    Qwen3.6-27B: Flagship-level coding in a 27B dense model, April 2026

    Qwen Team. Qwen3.6-27B: Flagship-level coding in a 27B dense model, April 2026

  36. [36]

    Qwen3.6-35B-A3B: Agentic coding power, now open to all, April 2026

    Qwen Team. Qwen3.6-35B-A3B: Agentic coding power, now open to all, April 2026

  37. [37]

    Solaris: Building a multiplayer video world model in minecraft.arXiv preprint arXiv:2602.22208, 2026

    Georgy Savva, Oscar Michel, Daohan Lu, Suppakit Waiwitlikhit, Timothy Meehan, Dhairya Mishra, Srivats Poddar, Jack Lu, and Saining Xie. Solaris: Building a multiplayer video world model in minecraft.arXiv preprint arXiv:2602.22208, 2026

  38. [38]

    Improved aesthetic predictor, 2022

    Christoph Schuhmann. Improved aesthetic predictor, 2022. GitHub repository

  39. [39]

    Consistency Models

    Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models.arXiv preprint arXiv:2303.01469, 2023

  40. [40]

    Transnet v2: An effective deep network architecture for fast shot transition detection

    Tomás Soucek and Jakub Lokoc. Transnet v2: An effective deep network architecture for fast shot transition detection. In Proceedings of the 32nd ACM International Conference on Multimedia, 2024

  41. [41]

    WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modeling

    Wenqiang Sun, Haiyu Zhang, Haoyuan Wang, Junta Wu, Zehan Wang, Zhenwei Wang, Yunhong Wang, Jun Zhang, Tengfei Wang, and Chunchao Guo. Worldplay: Towards long-term geometric consistency for real-time interactive world modeling. arXiv preprint arXiv:2512.14614, 2025

  42. [42]

    VidGen-1M: A Large-Scale Dataset for Text-to-video Generation

    Zhiyu Tan, Xiaomeng Yang, Luozheng Qin, and Hao Li. Vidgen-1m: A large-scale dataset for text-to-video generation.arXiv preprint arXiv:2408.02629, 2024

  43. [43]

    Dreamx-world 1.0: A general-purpose interactive world model.arXiv preprint arXiv:2606.16993, 2026

    DreamX Team, Yancheng Bai, Rui Chen, Xiangxiang Chu, Rujing Dang, Hao Dou, Bingjie Gao, Qiwen Gu, Siyu Hong, Jiachen Lei, et al. Dreamx-world 1.0: A general-purpose interactive world model.arXiv preprint arXiv:2606.16993, 2026

  44. [44]

    Advancing Open-source World Models

    Robbyant Team, Zelin Gao, Qiuyu Wang, Yanhong Zeng, Jiapeng Zhu, Ka Leong Cheng, Yixuan Li, Hanlin Wang, Yinghao Xu, Shuailei Ma, Yihang Chen, Jie Liu, Yansong Cheng, Yao Yao, Jiayi Zhu, Yihao Meng, Kecheng Zheng, Qingyan Bai, Jingye Chen, Zehong Shen, Yue Yu, Xing Zhu, Yujun Shen, and Hao Ouyang. Advancing open-source world models.arXiv preprint arXiv:26...

  45. [45]

    MAGI-1: Autoregressive Video Generation at Scale

    Hansi Teng, Hongyu Jia, Lei Sun, Lingzhi Li, Maolin Li, Mingqiu Tang, Shuai Han, Tianning Zhang, WQ Zhang, Weifeng Luo, et al. Magi-1: Autoregressive video generation at scale.arXiv preprint arXiv:2505.13211, 2025

  46. [46]

    WorldDirector: Building Controllable World Simulators with Persistent Dynamic Memory

    Hanlin Wang, Hao Ouyang, Qiuyu Wang, Wen Wang, Qingyan Bai, Ka Leong Cheng, Yue Yu, Yixuan Li, Yihao Meng, Zichen Liu, Yanhong Zeng, Yujun Shen, and Qifeng Chen. Worlddirector: Building controllable world simulators with persistent dynamic memory.arXiv preprint arXiv:2607.02517, 2026. 18

  47. [47]

    Spatialvid: A large-scale video dataset with spatial annotations.arXiv preprint arXiv:2509.09676, 2025

    Jiahao Wang, Yufeng Yuan, Rujie Zheng, Youtian Lin, Jian Gao, Lin-Zhuo Chen, Yajie Bao, Yi Zhang, Chang Zeng, Yanxi Zhou, et al. Spatialvid: A large-scale video dataset with spatial annotations.arXiv preprint arXiv:2509.09676, 2025

  48. [48]

    Koala-36m: A large-scale video dataset improving consistency between fine-grained conditions and video content

    Qiuheng Wang, Yukai Shi, Jiarong Ou, Rui Chen, Ke Lin, Jiahao Wang, Boyuan Jiang, Haotian Yang, Mingwu Zheng, Xin Tao, et al. Koala-36m: A large-scale video dataset improving consistency between fine-grained conditions and video content. In IEEE Conf. Comput. Vis. Pattern Recog., 2025

  49. [49]

    Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memory

    Zile Wang, Zexiang Liu, Jiaxing Li, Kaichen Huang, Baixin Xu, Fei Kang, Mengyin An, Peiyu Wang, Biao Jiang, Yichen Wei, et al. Matrix-game 3.0: Real-time and streaming interactive world model with long-horizon memory.arXiv preprint arXiv:2604.08995, 2026

  50. [50]

    A learning algorithm for continually running fully recurrent neural networks.Neural computation, 1989

    Ronald J Williams and David Zipser. A learning algorithm for continually running fully recurrent neural networks.Neural computation, 1989

  51. [51]

    Exploring video quality assessment on user generated contents from aesthetic and technical perspectives

    Haoning Wu, Erli Zhang, Liang Liao, Chaofeng Chen, Jingwen Hou, Annan Wang, Wenxiu Sun, Qiong Yan, and Weisi Lin. Exploring video quality assessment on user generated contents from aesthetic and technical perspectives. InProceedings of the IEEE/CVF international conference on computer vision, 2023

  52. [52]

    ActWorld: From Explorable to Interactive World Model via Action-Aware Memory

    Zhexiao Xiong, Yizhi Song, Hao Kang, Qing Yan, Liming Jiang, Jenson Yang, Zhoujie Fu, Stathi Fotiadis, Angtian Wang, Zichuan Liu, et al. Actworld: From explorable to interactive world model via action-aware memory.arXiv preprint arXiv:2606.17730, 2026

  53. [53]

    Understanding and improving layer normalization

    Jingjing Xu, Xu Sun, Zhiyuan Zhang, Guangxiang Zhao, and Junyang Lin. Understanding and improving layer normalization. InAdv. Neural Inform. Process. Syst., 2019

  54. [54]

    Aoe: Always-on egocentric human video collection for embodied ai

    Bowen Yang, Zishuo Li, Yang Sun, Changtao Miao, Yifan Yang, Man Luo, Xiaotong Yan, Feng Jiang, Jinchuan Shi, Yankai Fu, et al. Aoe: Always-on egocentric human video collection for embodied ai. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2026

  55. [55]

    LongLive: Real-time Interactive Long Video Generation

    Shuai Yang, Wei Huang, Ruihang Chu, Yicheng Xiao, Yuyang Zhao, Xianbang Wang, Muyang Li, Enze Xie, Yingcong Chen, Yao Lu, Song Han, and Yukang Chen. Longlive: Real-time interactive long video generation.arXiv preprint arXiv:2509.22622, 2025

  56. [56]

    Yuille, and Feng Wang

    Timing Yang, Sucheng Ren, Alan L. Yuille, and Feng Wang. Vimix-14m: A curated multi-source video-text dataset with long-form, high-quality captions and crawl-free access.ArXiv, abs/2511.18382, 2025

  57. [57]

    Infinity-rope: Action-controllable infinite video generation emerges from autoregressive self-rollout

    Hidir Yesiltepe, Tuna Meral, Adil Kaan Akan, Kaan Oktay, and Pinar Yanardag. Infinity-rope: Action-controllable infinite video generation emerges from autoregressive self-rollout. InIEEE Conf. Comput. Vis. Pattern Recog., 2026

  58. [58]

    One-step diffusion with distribution matching distillation

    Tianwei Yin, Michaël Gharbi, Richard Zhang, Eli Shechtman, Fredo Durand, William T Freeman, and Taesung Park. One-step diffusion with distribution matching distillation. InIEEE Conf. Comput. Vis. Pattern Recog., 2024

  59. [59]

    From slow bidirectional to fast autoregressive video diffusion models

    Tianwei Yin, Qiang Zhang, Richard Zhang, William T Freeman, Fredo Durand, Eli Shechtman, and Xun Huang. From slow bidirectional to fast autoregressive video diffusion models. InIEEE Conf. Comput. Vis. Pattern Recog., 2025

  60. [60]

    MuSS: A Large-Scale Dataset and Cinematic Narrative Benchmark for Multi-Shot Subject-to-Video Generation

    Haojie Zhang, Di Wu, Bingyan Liu, Linjie Zhong, Yuancheng Wei, Xingsong Ye, Nanqing Liu, and Yaling Liang. Muss: A large- scale dataset and cinematic narrative benchmark for multi-shot subject-to-video generation.arXiv preprint arXiv:2604.23789, 2026

  61. [61]

    minWM: A Full-Stack Open-Source Framework for Real-Time Interactive Video World Models

    Min Zhao, Hongzhou Zhu, Bokai Yan, Zihan Zhou, Yimin Chen, Wenqiang Sun, Kaiwen Zheng, Guande He, Xiao Yang, Chongxuan Li, et al. minwm: A full-stack open-source framework for real-time interactive video world models.arXiv preprint arXiv:2605.30263, 2026

  62. [62]

    Large scale diffusion distillation via score-regularized continuous-time consistency

    Kaiwen Zheng, Yuji Wang, Qianli Ma, Huayu Chen, Jintao Zhang, Yogesh Balaji, Jianfei Chen, Ming-Yu Liu, Jun Zhu, and Qinsheng Zhang. Large scale diffusion distillation via score-regularized continuous-time consistency. InInt. Conf. Learn. Represent., 2026

  63. [63]

    SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformer

    Haoyi Zhu, Haozhe Liu, Yuyang Zhao, Tian Ye, Junsong Chen, Jincheng Yu, Tong He, Song Han, and Enze Xie. Sana-wm: Efficient minute-scale world modeling with hybrid linear diffusion transformer.arXiv preprint arXiv:2605.15178, 2026

  64. [64]

    Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generation

    Hongzhou Zhu, Min Zhao, Guande He, Hang Su, Chongxuan Li, and Jun Zhu. Causal forcing: Autoregressive diffusion distillation done right for high-quality real-time interactive video generation.arXiv preprint arXiv:2602.02214, 2026. 19