{"work":{"id":"03ae1098-067d-4fea-90da-4ace0a031a03","openalex_id":null,"doi":null,"arxiv_id":"2604.08995","raw_key":null,"title":"Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memory","authors":null,"authors_text":null,"year":2026,"venue":"cs.CV","abstract":"With the advancement of interactive video generation, diffusion models have increasingly demonstrated their potential as world models. However, existing approaches still struggle to simultaneously achieve memory-enabled long-term temporal consistency and high-resolution real-time generation, limiting their applicability in real-world scenarios. To address this, we present Matrix-Game 3.0, a memory-augmented interactive world model designed for 720p real-time longform video generation. Building upon Matrix-Game 2.0, we introduce systematic improvements across data, model, and inference. First, we develop an upgraded industrial-scale infinite data engine that integrates Unreal Engine-based synthetic data, large-scale automated collection from AAA games, and real-world video augmentation to produce high-quality Video-Pose-Action-Prompt quadruplet data at scale. Second, we propose a training framework for long-horizon consistency: by modeling prediction residuals and re-injecting imperfect generated frames during training, the base model learns self-correction; meanwhile, camera-aware memory retrieval and injection enable the base model to achieve long horizon spatiotemporal consistency. Third, we design a multi-segment autoregressive distillation strategy based on Distribution Matching Distillation (DMD), combined with model quantization and VAE decoder pruning, to achieve efficient real-time inference. Experimental results show that Matrix-Game 3.0 achieves up to 40 FPS real-time generation at 720p resolution with a 5B model, while maintaining stable memory consistency over minute-long sequences. Scaling up to a 2x14B model further improves generation quality, dynamics, and generalization. Our approach provides a practical pathway toward industrial-scale deployable world models.","external_url":"https://arxiv.org/abs/2604.08995","cited_by_count":null,"metadata_source":"pith","metadata_fetched_at":"2026-07-09T07:56:04.713905+00:00","pith_arxiv_id":"2604.08995","created_at":"2026-05-12T03:26:19.171206+00:00","updated_at":"2026-07-09T07:56:04.713905+00:00","title_quality_ok":true,"display_title":"Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memory","render_title":"Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memory"},"hub":{"state":{"work_id":"03ae1098-067d-4fea-90da-4ace0a031a03","tier":"hub","tier_reason":"10+ Pith inbound or 1,000+ external citations","pith_inbound_count":22,"external_cited_by_count":null,"distinct_field_count":2,"first_pith_cited_at":"2026-05-11T04:16:41+00:00","last_pith_cited_at":"2026-07-08T15:33:25+00:00","author_build_status":"not_needed","summary_status":"needed","contexts_status":"needed","graph_status":"needed","ask_index_status":"not_needed","reader_status":"not_needed","recognition_status":"not_needed","updated_at":"2026-08-21T20:39:47.893942+00:00","tier_text":"hub"},"tier":"hub","role_counts":[{"context_role":"background","n":1}],"polarity_counts":[{"context_polarity":"background","n":1}],"runs":{},"summary":{},"graph":{},"authors":[]}}