Multiplayer Interactive World Models with Representation Autoencoders

Adam Jelley; Aditya Makkar; Adrien Ramanana Rahary; Alyx Liao; Am\'elie Royer; Anthony Hu; Chris Mulder; Eloi Alonso; Florian Laurent; Fredrik Nor\'en

REVIEW 2 major objections 7 minor 2 cited by

Multiplayer Interactive World Models with Representation Autoencoders

T0 review · 2 major / 7 minor · reviewed 2026-07-07 · glm-5.2

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Four-player world model runs real-time, stays stable for hours

desk verdict First real-time, four-player, action-conditioned generative world model for a continuous-physics 3D game. The core engineering result is solid and well-supported. read the letter →

arxiv 2607.05352 v1 pith:456GODI3 submitted 2026-07-06 cs.CV cs.AIcs.LG

Anthony Hu , Václav Volhejn , Adrien Ramanana Rahary , Chris Mulder , Aditya Makkar , Amélie Royer , Manu Orsini , Alyx Liao

show 19 more authors

Adam Jelley Eloi Alonso Florian Laurent Fredrik Norén James Swingos Jan Hünermann Kent Rollins Lucas Hosseini Matthieu Le Cauchois Maxim Peter Pim de Witte Tim Brown Vincent Micheli Moritz Böhle Gabriel de Marmiesse Viktoriia Sharmanska Lucia Specia Michael Black Patrick Pérez

This is my paper · ORCID

classification cs.CVcs.AIcs.LG

keywords worldmodelmultiplayerlatentdiffusionrepresentationautoencoderforcingactionconditioningRocketLeaguereal-timegeneration

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims the first interactive world model that conditions on the simultaneous action streams of four agents in a fast, physically complex environment. The system, called MIRA, is a 5-billion-parameter latent diffusion model trained on 10,000 hours of Rocket League gameplay. It renders all four players' perspectives at 20 frames per second on a single GPU and, despite being trained only on 4-second clips, produces rollouts that remain visually coherent and dynamically faithful for hours. The central mechanism is a combination of three design choices: a latent prediction space built on a frozen pretrained feature extractor (DINOv3), which prevents long-horizon drift; diffusion forcing, which trains the model on partially corrupted context so it can survive its own imperfect predictions at inference; and tiled multiplayer conditioning, which lets a single transformer jointly render all four views and attribute each player's actions to the correct car. The paper argues that these choices together yield a model that captures the game's underlying dynamics rather than merely memorizing visual patterns, supported by evidence that the model stays coherent under human control despite being trained only on bot play.

What carries the argument

A representation autoencoder that compresses video into a compact latent by combining a frozen DINOv3 feature extractor with a learned linear bottleneck (2x spatial, 2x temporal downsampling), paired with a flow-matching diffusion transformer that predicts future latents autoregressively. Each frame receives an independent noise level during training (diffusion forcing), and the four players' views are tiled into a single grid so spatial attention can keep them mutually consistent.

What would settle it

If the model's rollouts degrade significantly when driven by action sequences that differ substantially from the Nexto bot's behavioral distribution — for example, sustained inaction, unusual aerial maneuvers, or adversarial inputs designed to push cars into states the bot never produces — then the model would have demonstrated pattern matching to one policy rather than learning general dynamics.

Watch

Extended reading notes

Core claim

The paper's central result is that conditioning a latent diffusion world model on four simultaneous action streams, predicting in the latent space of a frozen pretrained feature extractor, and training with diffusion forcing yields real-time, multi-agent rollouts that stay stable far beyond the training horizon. The pretrained feature extractor is the load-bearing ingredient for long-horizon stability: codecs built on from-scratch extractors reconstruct video more sharply but drift substantially over sustained rollouts, while the pretrained-extractor latent stays flat. Diffusion forcing is the second pillar, keeping teacher-forced rollouts from collapsing past the 4-second training window. T

Load-bearing premise

The model is trained entirely on data from a single bot policy (Nexto) on three fixed maps, so its claims about learning the game's dynamics rest on the assumption that it has captured the underlying physics rather than memorizing one policy's behavioral patterns.

Editorial extensions

If this is right

If the design transfers, agents could be trained or evaluated inside learned simulators for multi-agent tasks without running the real environment, reducing the cost of reinforcement learning.
The finding that pretrained feature extractors prevent rollout drift suggests that the smoothness of the prediction space, not just reconstruction quality, is what makes a latent suitable for autoregressive generation.
The action recoverability ratio (ARR) provides a reusable protocol for measuring whether generative models actually obey control inputs, addressing a gap between visual fidelity and dynamical correctness.
The two-stage training recipe (pretrain single-player, then warm-start multiplayer) offers a practical path for scaling multi-agent conditioning without requiring the full multiplayer compute budget from the start.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the model's stability under human control is genuine (not just interpolation within the bot's behavioral distribution), it would imply the model has learned a generalizable dynamics model rather than a policy imitation, which would be a stronger claim than the paper's single-bot training data would guarantee.
The emergent 'theory of mind' for unconditioned players could be tested more rigorously by training on data from multiple distinct bot policies and checking whether the model can interpolate between their playing styles at inference.
The drift resistance of pretrained feature extractors may connect to the spectral smoothness of self-supervised representations: nearby states mapping to nearby latents would absorb prediction errors rather than amplifying them, a hypothesis that could be tested by measuring the Lipschitz constant of the latent mapping.
The fact that the model generalizes beyond its training action distribution (staying stable when all cars sit still, or under human play) partially addresses the single-policy concern, but a systematic out-of-distribution action evaluation would strengthen the dynamics-learning claim.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

read the letter

This paper delivers what it claims: a real-time, four-player, action-conditioned latent diffusion world model for Rocket League, running at 20 fps on a single B200. That is a genuine first. No prior work has conditioned a generative world model on multiple simultaneous action streams in a continuous-physics 3D environment. The systematic ablation work is the real strength — Tables 2–8 isolate each design choice (pixel vs. latent, pretrained vs. from-scratch features, diffusion vs. teacher forcing, codec components) with matched compute and clean controls. The ARR metric is validated against human judgment (Pearson r=0.84), and the game-state probe is a sensible way to test whether the model encodes physics rather than just appearance. Code, data, and a live demo are all released. This is reproducible work at a scale few labs can match, and the authors earn credit for that. The stress-test concern about long-horizon dynamical fidelity is valid but narrower than it sounds. The paper measures gFID out to 300 seconds (Figure 9) and measures the game-state probe and ARR at short horizons. It does not cross the two — no probe error as a function of rollout time. So the claim of dynamical fidelity over long horizons rests on an extrapolation: short-horizon physics accuracy plus long-horizon distributional stability. This is a real gap, but it is a gap in evaluation, not evidence of failure. The flat gFID curves, the live demo under human control, and the qualitative probe overlays all point in the right direction. A referee should ask for the probe error tracked over rollout time, but the central result does not collapse without it. The single-bot-policy training data (Nexto) is the other soft spot. The authors acknowledge it. The emergent “theory of mind” for unconditioned players (Section 6.8) could be Nexto reproduction rather than learned agent modeling, and the paper does not rigorously distinguish the two. This is secondary to the main contribution. This paper is for researchers working on generative world models, video prediction, or learned simulators. It deserves a serious referee. The main asks: (1) game-state probe error as a function of rollout horizon, (2) clearer separation of Nexto reproduction from genuine learned dynamics in the emergent-behavior claims.

Referee Report

2 major / 7 minor

Summary. This paper introduces MIRA, a 5B-parameter latent diffusion world model for four-player Rocket League that conditions on all players' simultaneous action streams, generates 20 fps in real time on a single B200 GPU, and remains stable over long horizons. The model predicts in the latent space of a representation autoencoder built on a frozen DINOv3-L feature extractor, trained with flow matching and diffusion forcing. The paper provides systematic ablations of the codec design (Tables 2–7, 21–25), the training objective (Table 8, Figures 9–11), the multiplayer conditioning scheme (Figure 14), and scaling behavior (Figure 16). It also introduces two targeted evaluation metrics: the Action Recoverability Ratio (ARR), validated against human judgment (Pearson r=0.84, Figure 12), and a game-state probe that reads physical quantities from the model's activations. The authors release their dataset, codebase, and a live demo.

Significance. This is a substantial contribution to interactive world modeling. The multiplayer conditioning design—tiling four views into a single grid with per-player action embeddings—is a clean and effective solution to multi-agent attribution. The systematic codec ablations (particularly Table 3 and Figure 7 showing that a frozen pretrained feature extractor is what prevents long-horizon drift, despite worse reconstruction) provide actionable design guidance for the field. The ARR metric and its validation against human preference studies is a valuable methodological contribution. The release of 10,000 hours of gameplay data with aligned physics state, full training/inference code, and a live interactive demo sets a high standard for reproducibility. The real-time inference system (Section 5) is well-engineered and documented.

major comments (2)

§6.2, §6.8: The paper claims rollouts are 'both visually coherent and dynamically faithful to the commanded actions' and 'stay stable far beyond the training horizon' (abstract, §1, §6.1). However, the two lines of evidence are measured at different horizons: distributional metrics (gFID/gFVD/gFDD) extend to 300 seconds (Figures 7, 9), while the dynamical metrics—game-state probe error (Figure 16a) and ARR (Figure 13)—are reported only as functions of model size and training step, respectively, not as functions of rollout horizon. The game-state probe overlay in Figure 21 is shown for a short rollout only. This leaves open whether dynamical fidelity degrades over long horizons even as distributional quality remains flat. The claim of long-horizon stability would be substantially strengthened by reporting ARR or probe error at multiple rollout horizons (e.g., 4s, 30s, 60s, 300s). As word,
§6.8: The 'theory of mind' claim for unconditioned players is presented as an emergent property, but since all training data is generated by a single bot policy (Nexto, §3.1), the model's behavior for dropped-action players may simply reproduce Nexto's policy distribution rather than learning a general agent model. The paper acknowledges this limitation in §3.1 but does not revisit it when making the 'theory of mind' claim in §6.8. The claim should be qualified: the model has learned to imitate the specific training policy for unconditioned players, which is a narrower result than general agent modeling. The live demo with human players (§6.8, Figure 18) provides partial counter-evidence for robustness to distribution shift, but does not directly test whether the unconditioned-player behavior generalizes beyond Nexto's strategy space.

minor comments (7)

§4.2: The adaptive gradient-norm balancing rule is described as reusing 'the gradient-norm balancing that VQ-GAN applies to its single adversarial term,' but VQ-GAN balances between reconstruction and discriminator losses, not between two perceptual terms and reconstruction. The analogy could be stated more precisely.
§6.3, Table 2: The pixel-space ARR is 'calibrated against real frames rather than a reconstruction' because pixel-space models have no codec. This makes the ARR comparison between latent and pixel space not apples-to-apples (the latent ARR divides by APrecon, the pixel ARR divides by APreal). The paper should note this asymmetry explicitly.
§6.7, Figure 16a: The ball-position probe error is reported in 'Unreal units' but the axis label says '×10³' without specifying the unit in the caption. Adding the unit (uu) to the caption would improve clarity.
§5: The paper states one full step takes 'roughly 70 ms end to end and produces two video frames (about 35 ms per frame),' which is within the 50 ms budget for 20 fps. It would be useful to report the variance or worst-case latency, since interactive applications are sensitive to tail latency, not just mean throughput.
§6.9: The failure case of the ball moving on its own when untouched is attributed to data imbalance. It would strengthen the analysis to report how frequently this occurs quantitatively, similar to the uncommanded boost/jump counts.
Figure 19 caption: The clock drift example shows the clock reading 4:54, 4:53, 4:53, 4:52, 4:54, 4:53 over five seconds. The text says 'it advances far too slowly and even ticks back up,' but the sequence also shows 4:52→4:54, which is a two-second jump forward followed by a one-second jump back. This pattern (both too slow and occasionally jumping) could be described more precisely.
References: Several cited works have 2026 dates (e.g., Siméoni et al. 2025 for DINOv3 is listed as arXiv:2508.10104, but other references like Tong et al. 2026, Singh et al. 2026, Hansen-Estruch et al. 2026 appear to be from 2026). The mixing of 2025 and 2026 dates should be verified for consistency.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the careful reading and the constructive feedback. Both major comments identify genuine gaps in our evaluation that we will address in revision. Below we respond point by point.

read point-by-point responses

Referee: §6.2, §6.8: Distributional metrics extend to 300s but dynamical metrics (ARR, game-state probe) are not reported as functions of rollout horizon. The claim of long-horizon stability would be strengthened by reporting ARR or probe error at multiple rollout horizons (e.g., 4s, 30s, 60s, 300s).

Authors: The referee is correct. Our distributional metrics (gFID/gFVD/gFDD) are tracked over rollout horizon up to 300 seconds (Figures 7, 9, 10), but our two dynamical metrics—ARR (Figure 13) and the game-state probe error (Figure 16a)—are reported only as functions of training step and model size, respectively, not as functions of rollout horizon. The game-state probe overlay in Figure 21 is shown for a short rollout only. This is a real gap: it leaves open the possibility that dynamical fidelity degrades over long horizons even as distributional quality remains flat, which would weaken our long-horizon stability claim. We will address this by computing both ARR and the game-state probe error at multiple rollout horizons (4s, 30s, 60s, 120s, 300s) on the flagship 5B model, conditioned on ground-truth actions, and adding the results as a new figure. We will also soften the abstract and §6.1 claims to specify that long-horizon stability has been verified for distributional quality and will be verified for dynamical fidelity in the revision. We note that the game-state probe is trained on real latents and applied to generated rollouts (§6.2), so extending it to multiple horizons is straightforward computationally; the main cost is running the probe over long rollouts, which we can do with existing infrastructure. revision: yes
Referee: §6.8: The 'theory of mind' claim for unconditioned players is presented as an emergent property, but since all training data is generated by a single bot policy (Nexto, §3.1), the model's behavior for dropped-action players may simply reproduce Nexto's policy distribution rather than learning a general agent model. The claim should be qualified.

Authors: The referee raises a valid concern. We acknowledge in §3.1 that using a single bot policy (Nexto) limits behavioral diversity, but we do not revisit this limitation when making the 'theory of mind' claim in §6.8. The referee is right that, since all four cars in every training match are driven by independent instances of the same Nexto policy, the model's behavior for unconditioned players most plausibly reflects imitation of Nexto's policy distribution learned from pixels, rather than a general agent model that would transfer to arbitrary player strategies. The live demo with human players (§6.8, Figure 18) provides partial counter-evidence—the model stays coherent under human control, which is outside the training action distribution—but this tests robustness to distribution shift in the conditioned player's actions, not whether the unconditioned-player behavior generalizes beyond Nexto's strategy space. We will revise §6.8 to qualify the claim explicitly: the model has learned to imitate the specific training policy for unconditioned players, which is a narrower result than general agent modeling. We will also add a forward reference to the §3.1 limitation at the point where the claim is made. We retain the observation that the model recovers complex decisions from pixels alone (the bot has access to privileged game state), but we will frame this as policy imitation from observation rather than 'theory of mind' in the general sense. revision: yes

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: derivation chain is self-contained with external benchmarks

full rationale

The paper's derivation chain is self-contained. The codec builds on a frozen, externally-developed feature extractor (DINOv3-L, Siméoni et al. 2025) and the representation autoencoder paradigm (Zheng et al. 2025, Singh et al. 2026), both external. The world model uses flow matching (Lipman et al. 2023) and diffusion forcing (Chen et al. 2024), also external. Evaluation metrics are independently grounded: gFID/gFVD use Inception-V3 and a 3D backbone; gFDD and ARR use a frozen DINOv3-B probe calibrated on real held-out data (0.84 mAP, Table 20) and validated against human judgment (Pearson r=0.84, Spearman ρ=0.93, Figure 12). The game-state probe is trained on real latents and tested on generated rollouts—a legitimate transfer test, not a self-referential loop. Self-citations (GAIA-1, GAIA-2, Ramanana Rahary et al.) appear only in related work for context, not as load-bearing premises. The P-DINO perceptual loss in codec training and the FDD evaluation metric both use DINOv3 features, but they serve different purposes (reconstruction loss vs. distributional distance) and use different model variants (DINOv3-L for training, DINOv3-B for evaluation), so this is not circular. No step in the derivation reduces to its own inputs by construction.

Assumptions & free parameters 9 free parameters · 4 assumptions · 2 invented entities

The free parameters are standard hyperparameters for a large-scale diffusion model, not ad hoc constants introduced to make a derivation work. The axioms are domain assumptions about the experimental setting (game as proxy, bot diversity, metric validity) rather than mathematical postulates. The invented entities (ARR, the codec) are empirically validated artifacts, not postulated physical objects.

free parameters (9)

Latent channels (C) = 32
Chosen as the bottleneck dimensionality; not derived from a principle.
Context window (T) = 20 latent frames
Fixed-size rolling window at inference; chosen for real-time constraints.
Perceptual loss weights (lambda_p, lambda_d) = Adaptively balanced
Gradient-norm matched to L1 loss; the balancing rule itself is a design choice from VQ-GAN.
Action dropout rate = 0.1 per step, 0.5 subset-drop
Probability of replacing a player's action embedding with an absent token during training.
Context noise std (inference) = 0.2
Default re-noising level for past latents at inference; chosen as a light default that rescues the fragile distilled codec.
Aggregated DINOv3 blocks = {11, 13, 15, 17, 19, 21, 23}
Hand-selected intermediate layers for feature aggregation.
Spatial/temporal downsampling = 2x2 spatial, 2x temporal
Chosen compression factors for the bottleneck.
Training clip length = 80 frames (4s)
Fixed training window length.
Flow-matching steps (inference) = 10
Default sampling steps for evaluation.

assumptions (4)

domain assumption Frozen DINOv3-L features provide a smooth, semantically meaningful latent space that prevents long-horizon rollout drift.
Invoked in Section 4.2 and supported empirically in Table 3 and Figure 7, but presented as an assumption about why pretrained features help rather than a derived result.
domain assumption Rocket League is a useful proxy for real-world physically dynamic multi-agent environments.
Stated in Section 1; the game is a video game simulation, not a real physical system, so transfer to real environments is assumed rather than demonstrated.
domain assumption The Action Recoverability Ratio (ARR) is a valid proxy for controllability.
Section 6.2 defines ARR and Section 6.5 validates it against human judgment (r=0.84), but it remains a proxy that measures action visibility, not causal fidelity.
domain assumption Bot-generated gameplay (Nexto) provides sufficient behavioral diversity for training a general world model.
Section 3.1 acknowledges this is a limitation; the model inherits one policy's style of play.

invented entities (2)

Action Recoverability Ratio (ARR) independent evidence
purpose: Metric measuring how faithfully generated rollouts obey commanded actions, by checking if actions are recoverable from generated video via a probe.
Validated against human preference (Figure 12, Pearson r=0.84, Spearman rho=0.93) and provides a falsifiable, quantitative handle on controllability.
Representation autoencoder codec with temporal downsampling independent evidence
purpose: Compresses video into a compact latent at 10 Hz built on frozen DINOv3 features, for stable real-time world modeling.
Ablated against pixel-space, from-scratch, and no-downsampling alternatives (Tables 2-5); the codec is a trained artifact, not a postulated entity.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Multiplayer Interactive World Models with Representation Autoencoders." pith.science (2026). https://pith.science/paper/456GODI3

@misc{pith2026260705352,
  author       = {Pith},
  title        = {Pith review of: Multiplayer Interactive World Models with Representation Autoencoders},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/456GODI3}},
  note         = {Machine review of arXiv:2607.05352}
}

read the original abstract

We introduce the first multiplayer world model for highly dynamic environments governed by complex physical interactions. Whereas single-player world models treat the other agents as part of the environment, ours conditions on the action streams of multiple agents, learning to attribute changes in the scene to the correct player and to stay coherent under arbitrary combinations of their actions. We study this problem in the game of Rocket League, where players compete and cooperate under fast, tightly coupled dynamics. Trained on 10,000 hours of gameplay collected with publicly available bots, our 5-billion-parameter latent diffusion model generates four-player matches in real time, producing 20 frames per second on a single Nvidia B200 GPU. Although trained only on short clips, its rollouts stay stable far beyond the training horizon: distributional quality holds steady out to five minutes, the longest horizon we measure, and in practice we observe rollouts continuing for hours with no sign of collapse. We systematically investigate the central design choices: the video codec, the generative objective, and the multiplayer conditioning scheme. In addition, we characterize how behavior changes with model and data scale, including the capabilities that emerge and the failure modes that persist. We further develop targeted evaluations that probe the model's physical understanding rather than visual appearance alone. To support continued research on multiplayer world models, we release our dataset, our full training and inference codebase, and a live demo.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Population-Scalable Multi-Agent World Modeling
cs.CV 2026-08 conditional novelty 6.0 of 10

Khora decouples world-state evolution from visual rendering through a shared STBoard and fixed-dimensional per-view renderers, enabling inference-time addition and removal of agents without retraining.
MASS: Multiplayer World Models with Authoritative Shared State
cs.CV 2026-08 conditional novelty 6.0 of 10

MASS predicts an explicit typed world state once per tick, then renders any number of camera views from that same state, improving state recovery and cross-view consistency on a Snake benchmark.

Reference graph

Works this paper leans on

280 extracted references · 280 canonical work pages · cited by 2 Pith papers

[1]

Advances in Neural Information Processing Systems (NeurIPS) , year =

Recurrent World Models Facilitate Policy Evolution , author =. Advances in Neural Information Processing Systems (NeurIPS) , year =

work page
[2]

International Conference on Machine Learning (ICML) , year =

Learning Latent Dynamics for Planning from Pixels , author =. International Conference on Machine Learning (ICML) , year =

work page
[3]

International Conference on Learning Representations (ICLR) , year =

Dream to Control: Learning Behaviors by Latent Imagination , author =. International Conference on Learning Representations (ICLR) , year =

work page
[4]

International Conference on Learning Representations (ICLR) , year =

Mastering Atari with Discrete World Models , author =. International Conference on Learning Representations (ICLR) , year =

work page
[5]

Nature , year =

Mastering Diverse Domains through World Models , author =. Nature , year =

work page
[7]

International Conference on Learning Representations (ICLR) , year =

Transformers are Sample-Efficient World Models , author =. International Conference on Learning Representations (ICLR) , year =

work page
[8]

International Conference on Learning Representations (ICLR) , year =

Transformer-based World Models Are Happy With 100k Interactions , author =. International Conference on Learning Representations (ICLR) , year =

work page
[9]

Zhang, Weipu and Wang, Gang and Sun, Jian and Yuan, Yetian and Huang, Gao , booktitle =

work page

Show all 280 references

[10]

International Conference on Machine Learning (ICML) , year =

Temporal Difference Learning for Model Predictive Control , author =. International Conference on Machine Learning (ICML) , year =
[11]

Hansen, Nicklas and Su, Hao and Wang, Xiaolong , booktitle =
[12]

2022 , note =

A Path Towards Autonomous Machine Intelligence , author =. 2022 , note =

2022
[13]

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year =

Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture , author =. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year =
[14]

Transactions on Machine Learning Research (TMLR) , year =

Revisiting Feature Prediction for Learning Visual Representations from Video , author =. Transactions on Machine Learning Research (TMLR) , year =
[15]

Assran, Mahmoud and others , journal =
[16]

Zhou, Gaoyue and Pan, Hengkai and LeCun, Yann and Pinto, Lerrel , booktitle =
[17]

Advances in Neural Information Processing Systems (NeurIPS) , year =

Video Diffusion Models , author =. Advances in Neural Information Processing Systems (NeurIPS) , year =
[18]

and Poole, Ben and Norouzi, Mohammad and Fleet, David J

Ho, Jonathan and Chan, William and Saharia, Chitwan and Whang, Jay and Gao, Ruiqi and Gritsenko, Alexey and Kingma, Diederik P. and Poole, Ben and Norouzi, Mohammad and Fleet, David J. and Salimans, Tim , journal =
[19]

Singer, Uriel and Polyak, Adam and Hayes, Thomas and Yin, Xi and An, Jie and Zhang, Songyang and Hu, Qiyuan and Yang, Harry and Ashual, Oron and Gafni, Oran and Parikh, Devi and Gupta, Sonal and Taigman, Yaniv , booktitle =
[21]

2024 , note =

Video Generation Models as World Simulators , author =. 2024 , note =

2024
[22]

Yang, Zhuoyi and Teng, Jiayan and Zheng, Wendi and Ding, Ming and Huang, Shiyu and Xu, Jiazheng and Yang, Yuanming and Hong, Wenyi and Zhang, Xiaohan and Feng, Guanyu and others , booktitle =
[23]

Kong, Weijie and others , journal =
[24]

HaCohen, Yoav and others , journal =
[25]

International Conference on Learning Representations (ICLR) , year =

Vision Transformers Need Registers , author =. International Conference on Learning Representations (ICLR) , year =
[26]

Advances in Neural Information Processing Systems (NeurIPS) , year =

Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free , author =. Advances in Neural Information Processing Systems (NeurIPS) , year =
[28]

Shazeer, Noam , journal =
[29]

Conference on Empirical Methods in Natural Language Processing (EMNLP) , year =

Ainslie, Joshua and Lee-Thorp, James and de Jong, Michiel and Zemlyanskiy, Yury and Lebr. Conference on Empirical Methods in Natural Language Processing (EMNLP) , year =
[30]

Findings of the Association for Computational Linguistics: EMNLP , year =

Query-Key Normalization for Transformers , author =. Findings of the Association for Computational Linguistics: EMNLP , year =
[31]

International Conference on Machine Learning (ICML) , year =

Scaling Vision Transformers to 22 Billion Parameters , author =. International Conference on Machine Learning (ICML) , year =
[32]

Su, Jianlin and Lu, Yu and Pan, Shengfeng and Murtadha, Ahmed and Wen, Bo and Liu, Yunfeng , journal =
[33]

International Conference on Machine Learning (ICML) , year =

Is Space-Time Attention All You Need for Video Understanding? , author =. International Conference on Machine Learning (ICML) , year =
[34]

IEEE/CVF International Conference on Computer Vision (ICCV) , year =

Arnab, Anurag and Dehghani, Mostafa and Heigold, Georg and Sun, Chen and Lu. IEEE/CVF International Conference on Computer Vision (ICCV) , year =
[35]

Polyak, Adam and others , journal =
[36]

Bar-Tal, Omer and Chefer, Hila and Tov, Omer and Herrmann, Charles and Paiss, Roni and Zada, Shiran and Ephrat, Ariel and Hur, Junhwa and Liu, Guanghui and Raj, Amit and others , booktitle =
[37]

International Conference on Machine Learning (ICML) , year =

Kondratyuk, Dan and Yu, Lijun and Gu, Xiuye and Lezama, Jos. International Conference on Machine Learning (ICML) , year =
[38]

European Conference on Computer Vision (ECCV) , year =

Photorealistic Video Generation with Diffusion Models , author =. European Conference on Computer Vision (ECCV) , year =
[39]

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year =

High-Resolution Image Synthesis with Latent Diffusion Models , author =. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year =
[40]

IEEE/CVF International Conference on Computer Vision (ICCV) , year =

Scalable Diffusion Models with Transformers , author =. IEEE/CVF International Conference on Computer Vision (ICCV) , year =
[41]

Advances in Neural Information Processing Systems (NeurIPS) , year =

Denoising Diffusion Probabilistic Models , author =. Advances in Neural Information Processing Systems (NeurIPS) , year =
[42]

International Conference on Learning Representations (ICLR) , year =

Flow Matching for Generative Modeling , author =. International Conference on Learning Representations (ICLR) , year =
[43]

International Conference on Learning Representations (ICLR) , year =

Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow , author =. International Conference on Learning Representations (ICLR) , year =
[44]

and Boffi, Nicholas M

Ma, Nanye and Goldstein, Mark and Albergo, Michael S. and Boffi, Nicholas M. and Vanden-Eijnden, Eric and Xie, Saining , booktitle =
[45]

International Conference on Learning Representations (ICLR) , year =

Diffusion Models Are Real-Time Game Engines , author =. International Conference on Learning Representations (ICLR) , year =
[46]

Bruce, Jake and Dennis, Michael and Edwards, Ashley and Parker-Holder, Jack and Shi, Yuge and Hughes, Edward and Lai, Matthew and Mavalankar, Aditi and Steigerwald, Richie and Apps, Chris and others , booktitle =
[47]

Diffusion for World Modeling: Visual Details Matter in

Alonso, Eloi and Jelley, Adam and Micheli, Vincent and Kanervisto, Anssi and Storkey, Amos and Pearce, Tim and Fleuret, Fran. Diffusion for World Modeling: Visual Details Matter in. Advances in Neural Information Processing Systems (NeurIPS) , year =
[48]

Che, Haoxuan and He, Xuanhua and Liu, Quande and Jin, Cheng and Chen, Hao , booktitle =
[49]

Feng, Ruili and Zhang, Han and Yang, Zhantao and Xiao, Jie and Shu, Zhilei and Liu, Zhiheng and Zheng, Andy and Huang, Yukun and Liu, Yu and Zhang, Hongyang , journal =. The
[50]

Zhang, Yifan and Wei, Chunli and Wu, Dongdong and He, Xianglong and Xu, Yiqun and Lyu, Xinjie and Zhang, Yongchao and Liu, Hao and Chen, Yang and others , journal =
[51]

He, Xianglong and Zhang, Chunli and Wu, Dongdong and Zhang, Yifan and Xu, Yiqun and others , journal =
[52]

2026 , howpublished=

Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memory , author=. 2026 , howpublished=

2026
[53]

Guo, Junliang and Ye, Yang and He, Tianyu and Wu, Haoyu and Jiang, Yushu and Pearce, Tim and Bian, Jiang , journal =
[54]

Fan, Linxi and Wang, Guanzhi and Jiang, Yunfan and Mandlekar, Ajay and Yang, Yuncong and Zhu, Haoyi and Tang, Andrew and Huang, De-An and Zhu, Yuke and Anandkumar, Anima , booktitle =
[55]

Li, Jiaqi and Zhang, Junshu and Jiang, Boyuan and Wang, Yuxuan and Zhao, Yujie and others , journal =
[56]

Learning to Simulate Dynamic Environments with

Kim, Seung Wook and Zhou, Yuhao and Philion, Jonah and Torralba, Antonio and Fidler, Sanja , booktitle =. Learning to Simulate Dynamic Environments with
[57]

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year =

Playable Video Generation , author =. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year =
[58]

arXiv preprint arXiv:2503.17359 , year =

Position: Interactive Generative Video as Next-Generation Game Engine , author =. arXiv preprint arXiv:2503.17359 , year =

arXiv
[59]

Advances in Neural Information Processing Systems (NeurIPS) , year =

Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion , author =. Advances in Neural Information Processing Systems (NeurIPS) , year =
[60]

International Conference on Machine Learning (ICML) , year =

History-Guided Video Diffusion , author =. International Conference on Machine Learning (ICML) , year =
[61]

Advances in Neural Information Processing Systems (NeurIPS) , year =

Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion , author =. Advances in Neural Information Processing Systems (NeurIPS) , year =
[62]

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year =

From Slow Bidirectional to Fast Autoregressive Video Diffusion Models , author =. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year =
[63]

International Conference on Learning Representations (ICLR) , year =

One Step Diffusion via Shortcut Models , author =. International Conference on Learning Representations (ICLR) , year =
[64]

International Conference on Machine Learning (ICML) , year =

Consistency Models , author =. International Conference on Machine Learning (ICML) , year =
[65]

International Conference on Learning Representations (ICLR) , year =

Improved Techniques for Training Consistency Models , author =. International Conference on Learning Representations (ICLR) , year =
[68]

Advances in Neural Information Processing Systems (NeurIPS) , year =

Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks , author =. Advances in Neural Information Processing Systems (NeurIPS) , year =
[69]

Advances in Neural Information Processing Systems (NeurIPS) , year =

Professor Forcing: A New Algorithm for Training Recurrent Networks , author =. Advances in Neural Information Processing Systems (NeurIPS) , year =
[70]

International Conference on Machine Learning (ICML) , year =

Rolling Diffusion Models , author =. International Conference on Machine Learning (ICML) , year =
[71]

Kim, Jihwan and Kang, Junoh and Choi, Jinyoung and Han, Bohyung , booktitle =
[73]

Nature , volume =

World and Human Action Models towards Gameplay Ideation , author =. Nature , volume =
[74]

Pondaven, Alexander and Wu, Haoyu and Gilitschenski, Igor and Torr, Philip and Tulyakov, Sergey and Pizzati, Fabio and Siarohin, Aliaksandr , journal =
[75]

Liu, Fangfu and He, Kai and Ren, Xuanchi and others , journal =
[76]

Savva, Georgy and Michel, Oscar and Xie, Saining and others , journal =
[77]

Wu, Haoyu and Yu, Jiwen and Zou, Yingtian and Liu, Xihui , journal =
[78]

Po, Ryan and Zhang, Kai and Hertz, Amir and Wetzstein, Gordon and Wadhwa, Neal and Ruiz, Nataniel , journal =
[79]

Hu, Teng and others , journal =
[80]

Zhu, Shangwen and Peng, Yiran and Feng, Ruili and others , journal =
[81]

Liu, Andrew and Tucker, Richard and Jampani, Varun and Makadia, Ameesh and Snavely, Noah and Kanazawa, Angjoo , booktitle =
[82]

Li, Zhengqi and Wang, Qianqian and Snavely, Noah and Kanazawa, Angjoo , booktitle =
[83]

Liu, Ruoshi and Wu, Rundi and Van Hoorick, Basile and Tokmakov, Pavel and Zakharov, Sergey and Vondrick, Carl , booktitle =
[84]

Sargent, Kyle and Li, Zizhang and Shah, Tanmay and Herrmann, Charles and Yu, Hong-Xing and Zhang, Yunzhi and Chan, Eric Ryan and Lagun, Dmitry and Fei-Fei, Li and Sun, Deqing and Wu, Jiajun , booktitle =
[85]

Koh, Jing Yu and Lee, Honglak and Yang, Yinfei and Baldridge, Jason and Anderson, Peter , booktitle =
[86]

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year =

Navigation World Models , author =. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year =
[87]

Ma, Zehong and others , journal =
[89]

2024 , note =

Generating Worlds , author =. 2024 , note =

2024
[90]

He, Hao and Xu, Yinghao and Guo, Yuwei and Wetzstein, Gordon and Dai, Bo and Li, Hongsheng and Yang, Ceyuan , journal =
[91]

IEEE/CVF International Conference on Computer Vision (ICCV) , year =

Emerging Properties in Self-Supervised Vision Transformers , author =. IEEE/CVF International Conference on Computer Vision (ICCV) , year =
[92]

Transactions on Machine Learning Research (TMLR) , year =

Oquab, Maxime and Darcet, Timoth. Transactions on Machine Learning Research (TMLR) , year =
[94]

Peng, Zhiliang and Dong, Li and Bao, Hangbo and Ye, Qixiang and Wei, Furu , journal =
[95]

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year =

Masked Autoencoders Are Scalable Vision Learners , author =. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year =
[96]

IEEE/CVF International Conference on Computer Vision (ICCV) , year =

Sigmoid Loss for Language Image Pre-Training , author =. IEEE/CVF International Conference on Computer Vision (ICCV) , year =
[97]

Tschannen, Michael and Gritsenko, Alexey and Wang, Xiao and Naeem, Muhammad Ferjad and Alabdulmohsin, Ibrahim and others , journal =
[98]

arXiv preprint arXiv:2510.11690 , year =

Diffusion Transformers with Representation Autoencoders , author =. arXiv preprint arXiv:2510.11690 , year =

arXiv
[99]

CVPR , year =

Taming Transformers for High-Resolution Image Synthesis , author =. CVPR , year =
[101]

arXiv preprint arXiv:2501.09755 , year =

Learnings from Scaling Visual Tokenizers for Reconstruction and Generation , author =. arXiv preprint arXiv:2501.09755 , year =

arXiv
[102]

Hansen-Estruch, Philippe and Chen, Jiahui and Ramanujan, Vivek and Zohar, Orr and Ping, Yan and Sinha, Animesh and Georgopoulos, Markos and Schoenfeld, Edgar and Hou, Ji and Juefei-Xu, Felix and Vishwanath, Sriram and Thabet, Ali , journal =
[103]

Making Reconstruction

Xu, Tongda and others , journal =. Making Reconstruction
[104]

Generation: Taming Optimization Dilemma in Latent Diffusion Models , author =

Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Models , author =. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year =
[105]

International Conference on Machine Learning (ICML) , year =

Masked Autoencoders Are Effective Tokenizers for Diffusion Models , author =. International Conference on Machine Learning (ICML) , year =
[106]

Chen, Junyu and Zou, Dongyun and He, Wenkun and Chen, Junsong and Xie, Enze and Han, Song and Cai, Han , booktitle =
[107]

arXiv preprint arXiv:2507.15856 , year =

Latent Denoising Makes Good Tokenizers , author =. arXiv preprint arXiv:2507.15856 , year =
[108]

Kouzelis, Theodoros and Kakogeorgiou, Ioannis and Gidaris, Spyros and Komodakis, Nikos , booktitle =
[112]

Bi, Tianci and Zhang, Xiaoyi and Lu, Yan and Zheng, Nanning , journal =
[115]

Heusel, Martin and Ramsauer, Hubert and Unterthiner, Thomas and Nessler, Bernhard and Hochreiter, Sepp , booktitle =
[116]

Improved Techniques for Training

Salimans, Tim and Goodfellow, Ian and Zaremba, Wojciech and Cheung, Vicki and Radford, Alec and Chen, Xi , booktitle =. Improved Techniques for Training
[117]

IEEE Transactions on Image Processing , volume =

Image Quality Assessment: From Error Visibility to Structural Similarity , author =. IEEE Transactions on Image Processing , volume =
[118]

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year =

The Unreasonable Effectiveness of Deep Features as a Perceptual Metric , author =. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year =
[120]

Huang, Ziqi and He, Yinan and Yu, Jiashuo and Zhang, Fan and Si, Chenyang and Jiang, Yuming and Zhang, Yuanhan and Wu, Tianxing and Jin, Qingyang and Chanpaisit, Nattapol and others , booktitle =
[122]

Bansal, Hritik and Lin, Zongyu and Xie, Tianyi and Zong, Zeshun and Yarom, Michal and Bitton, Yonatan and Jiang, Chenfanfu and Sun, Yizhou and Chang, Kai-Wei and Grover, Aditya , journal =
[123]

International Conference on Machine Learning (ICML) , year =

Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generation , author =. International Conference on Machine Learning (ICML) , year =
[124]

Duan, Haoyi and Guo, Hong-Xing and Zhao, Xiaoshuai and Wu, Jiajun and others , booktitle =
[125]

Li, Dacheng and Zhang, Yunhao and Lin, Ji and Xie, Enze and others , journal =
[126]

International Conference on Learning Representations (ICLR) Workshop , year =

Understanding Intermediate Layers Using Linear Classifier Probes , author =. International Conference on Learning Representations (ICLR) Workshop , year =
[127]

International Conference on Learning Representations (ICLR) , year =

Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task , author =. International Conference on Learning Representations (ICLR) , year =
[128]

Maes, Lucas and Le Lidec, Quentin and Scieur, Damien and LeCun, Yann and Balestriero, Randall , journal =
[129]

International Conference on Machine Learning (ICML) , year =

Efficient World Models with Context-Aware Tokenization , author =. International Conference on Machine Learning (ICML) , year =
[130]

Hu, Anthony and Russell, Lloyd and Yeo, Hudson and Murez, Zak and Fedoseev, George and Kendall, Alex and Shotton, Jamie and Corrado, Gianluca , journal =
[131]

Russell, Lloyd and Hu, Anthony and Bertoni, Lorenzo and Fedoseev, George and Shotton, Jamie and Arani, Elahe and Corrado, Gianluca , journal =
[132]

Advances in Neural Information Processing Systems (NeurIPS) , year =

Model-Based Imitation Learning for Urban Driving , author =. Advances in Neural Information Processing Systems (NeurIPS) , year =
[133]

Wang, Xiaofeng and Zhu, Zheng and Huang, Guan and Chen, Xinze and Zhu, Jiagang and Lu, Jiwen , booktitle =
[134]

Gao, Shenyuan and Yang, Jiazhi and Chen, Li and Chitta, Kashyap and Qiu, Yihang and Geiger, Andreas and Zhang, Jun and Li, Hongyang , booktitle =
[135]

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year =

Generalized Predictive Model for Autonomous Driving , author =. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year =
[136]

Bartoccioni, Florent and Ramzi, Elias and Besnier, Victor and others , journal =
[137]

arXiv:2410.11081 , year=

Simplifying, Stabilizing and Scaling Continuous-Time Consistency Models , author=. arXiv:2410.11081 , year=

arXiv
[138]

Necto/Nexto: A Rocket League bot trained with deep reinforcement learning , year =

Rolv-Arild Braaten and. Necto/Nexto: A Rocket League bot trained with deep reinforcement learning , year =
[139]

2024 , howpublished =

2024
[140]

2021 , howpublished =

2021
[141]

2016 , howpublished =

Chris Mulder and. 2016 , howpublished =

2016
[142]

Google AI , year=

Welcome to the era of experience , author=. Google AI , year=
[143]

Nature , year =

Mastering diverse control tasks through world models , author =. Nature , year =
[144]

2025 , eprint =

WorldGym: World Model as An Environment for Policy Evaluation , author =. 2025 , eprint =

2025
[145]

2025 , eprint =

Evaluating. 2025 , eprint =

2025
[146]

Mastering

Schrittwieser, Julian and Antonoglou, Ioannis and Hubert, Thomas and Simonyan, Karen and Sifre, Laurent and Schmitt, Simon and Guez, Arthur and Lockhart, Edward and Hassabis, Demis and Graepel, Thore and Lillicrap, Timothy and Silver, David , journal =. Mastering
[147]

Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics (AISTATS) , editor =

A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning , author =. Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics (AISTATS) , editor =
[148]

2024 , publisher =

Multi-Agent Reinforcement Learning: Foundations and Modern Approaches , author =. 2024 , publisher =

2024
[149]

International Conference on Learning Representations (ICLR) , year =

An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale , author =. International Conference on Learning Representations (ICLR) , year =
[150]

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year=

Worldgen: From text to traversable and interactive 3d worlds , author=. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , year=
[154]

2022 , eprint =

Improving Multimodal Interactive Agents with Reinforcement Learning from Human Feedback , author =. 2022 , eprint =

2022
[156]

Zhang, Xin and Zhang, Dong and Li, Shimin and Zhou, Yaqian and Qiu, Xipeng , booktitle =
[157]

Chen, Sanyuan and Wang, Chengyi and Chen, Zhengyang and Wu, Yu and Liu, Shujie and Chen, Zhuo and Li, Jinyu and Kanda, Naoyuki and Yoshioka, Takuya and Xiao, Xiong and others , journal =
[158]

Improving multimodal interactive agents with reinforcement learning from human feedback, 2022

Josh Abramson, Arun Ahuja, Federico Carnevale, Petko Georgiev, Alex Goldin, Alden Hung, Jessica Landon, Jirka Lhotka, Timothy Lillicrap, Alistair Muldal, George Powell, Adam Santoro, Guy Scully, Sanjana Srivastava, Tamara von Glehn, Greg Wayne, Nathaniel Wong, Chen Yan, and Ru...

2022
[159]

GQA : Training generalized multi-query transformer models from multi-head checkpoints

Joshua Ainslie, James Lee-Thorp, Michiel de Jong, Yury Zemlyanskiy, Federico Lebr \'o n, and Sumit Sanghai. GQA : Training generalized multi-query transformer models from multi-head checkpoints. In Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023

2023
[160]

Understanding intermediate layers using linear classifier probes

Guillaume Alain and Yoshua Bengio. Understanding intermediate layers using linear classifier probes. In International Conference on Learning Representations (ICLR) Workshop, 2017

2017
[161]

Albrecht, Filippos Christianos, and Lukas Sch \"a fer

Stefano V. Albrecht, Filippos Christianos, and Lukas Sch \"a fer. Multi-Agent Reinforcement Learning: Foundations and Modern Approaches. MIT Press, 2024

2024
[162]

Diffusion for world modeling: Visual details matter in Atari

Eloi Alonso, Adam Jelley, Vincent Micheli, Anssi Kanervisto, Amos Storkey, Tim Pearce, and Fran c ois Fleuret. Diffusion for world modeling: Visual details matter in Atari . In Advances in Neural Information Processing Systems (NeurIPS), 2024

2024
[163]

Jason Ansel, Edward Yang, Horace He, Natalia Gimelshein, Animesh Jain, Michael Voznesensky, Bin Bao, Peter Bell, David Berard, Evgeni Burovski, Geeta Chauhan, Anjali Chourdia, Will Constable, Alban Desmaison, Zachary DeVito, Elias Ellison, Will Feng, Jiong Gong, Michael Gschwi...

2024 doi
[164]

ViViT : A video vision transformer

Anurag Arnab, Mostafa Dehghani, Georg Heigold, Chen Sun, Mario Lu c i \'c , and Cordelia Schmid. ViViT : A video vision transformer. In IEEE/CVF International Conference on Computer Vision (ICCV), 2021

2021
[165]

Self-supervised learning from images with a joint-embedding predictive architecture

Mahmoud Assran, Quentin Duval, Ishan Misra, Piotr Bojanowski, Pascal Vincent, Michael Rabbat, Yann LeCun, and Nicolas Ballas. Self-supervised learning from images with a joint-embedding predictive architecture. In IEEE/CVF Conference on Computer Vision and Pattern Recognition ...

2023
[166]

V-JEPA 2 : Self-supervised video models enable understanding, prediction and planning

Mahmoud Assran et al. V-JEPA 2 : Self-supervised video models enable understanding, prediction and planning. arXiv preprint arXiv:2506.09985, 2025

2025 arXiv
[167]

VideoPhy : Evaluating physical commonsense for video generation

Hritik Bansal, Zongyu Lin, Tianyi Xie, Zeshun Zong, Michal Yarom, Yonatan Bitton, Chenfanfu Jiang, Yizhou Sun, Kai-Wei Chang, and Aditya Grover. VideoPhy : Evaluating physical commonsense for video generation. arXiv preprint arXiv:2406.03520, 2024

2024 arXiv
[168]

Navigation world models

Amir Bar, Gaoyue Zhou, Danny Tran, Trevor Darrell, and Yann LeCun. Navigation world models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025

2025
[169]

Lumiere : A space-time diffusion model for video generation

Omer Bar-Tal, Hila Chefer, Omer Tov, Charles Herrmann, Roni Paiss, Shiran Zada, Ariel Ephrat, Junhwa Hur, Guanghui Liu, Amit Raj, et al. Lumiere : A space-time diffusion model for video generation. In SIGGRAPH Asia, 2024

2024
[170]

Revisiting feature prediction for learning visual representations from video

Adrien Bardes, Quentin Garrido, Jean Ponce, Xinlei Chen, Michael Rabbat, Yann LeCun, Mahmoud Assran, and Nicolas Ballas. Revisiting feature prediction for learning visual representations from video. Transactions on Machine Learning Research (TMLR), 2024. arXiv:2404.08471

2024 arXiv
[171]

VaViM and VaVAM : Autonomous driving through video generative modeling

Florent Bartoccioni, Elias Ramzi, Victor Besnier, et al. VaViM and VaVAM : Autonomous driving through video generative modeling. arXiv preprint arXiv:2502.15672, 2025

2025 arXiv
[172]

Scheduled sampling for sequence prediction with recurrent neural networks

Samy Bengio, Oriol Vinyals, Navdeep Jaitly, and Noam Shazeer. Scheduled sampling for sequence prediction with recurrent neural networks. In Advances in Neural Information Processing Systems (NeurIPS), 2015

2015
[173]

Is space-time attention all you need for video understanding? In International Conference on Machine Learning (ICML), 2021

Gedas Bertasius, Heng Wang, and Lorenzo Torresani. Is space-time attention all you need for video understanding? In International Conference on Machine Learning (ICML), 2021

2021
[174]

VFM-VAE : Vision foundation models can be good tokenizers for latent diffusion models

Tianci Bi, Xiaoyi Zhang, Yan Lu, and Nanning Zheng. VFM-VAE : Vision foundation models can be good tokenizers for latent diffusion models. arXiv preprint arXiv:2510.18457, 2025

2025 arXiv
[175]

Stable video diffusion: Scaling latent video diffusion models to large datasets

Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127, 2023

2023 arXiv
[176]

Boffi, Michael S

Nicholas M. Boffi, Michael S. Albergo, and Eric Vanden-Eijnden. How to build a consistency model: Learning flow maps via self-distillation. arXiv preprint arXiv:2505.18825, 2025

2025
[177]

Necto/nexto: A rocket league bot trained with deep reinforcement learning

Rolv-Arild Braaten and Necto contributors . Necto/nexto: A rocket league bot trained with deep reinforcement learning. https://github.com/Rolv-Arild/Necto, 2022

2022
[178]

Video generation models as world simulators, 2024

Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, et al. Video generation models as world simulators, 2024. OpenAI technical report, https://openai.com/index/video-generation-models-as-world-simulators/

2024
[179]

Genie : Generative interactive environments

Jake Bruce, Michael Dennis, Ashley Edwards, Jack Parker-Holder, Yuge Shi, Edward Hughes, Matthew Lai, Aditi Mavalankar, Richie Steigerwald, Chris Apps, et al. Genie : Generative interactive environments. In International Conference on Machine Learning (ICML), 2024

2024
[180]

Emerging properties in self-supervised vision transformers

Mathilde Caron, Hugo Touvron, Ishan Misra, Herv \'e J \'e gou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In IEEE/CVF International Conference on Computer Vision (ICCV), 2021

2021
[181]

GameGen-X : Interactive open-world game video generation

Haoxuan Che, Xuanhua He, Quande Liu, Cheng Jin, and Hao Chen. GameGen-X : Interactive open-world game video generation. In International Conference on Learning Representations (ICLR), 2025

2025
[182]

Diffusion forcing: Next-token prediction meets full-sequence diffusion

Boyuan Chen, Diego Mart \' Mons \'o , Yilun Du, Max Simchowitz, Russ Tedrake, and Vincent Sitzmann. Diffusion forcing: Next-token prediction meets full-sequence diffusion. In Advances in Neural Information Processing Systems (NeurIPS), 2024

2024
[183]

Masked autoencoders are effective tokenizers for diffusion models

Hao Chen, Yujin Han, Fangyi Chen, Xiang Li, Yidong Wang, Jindong Wang, Ze Wang, Zicheng Liu, Difan Zou, and Bhiksha Raj. Masked autoencoders are effective tokenizers for diffusion models. In International Conference on Machine Learning (ICML), 2025 a

2025
[184]

DC-AE 1.5 : Accelerating diffusion model convergence with structured latent space

Junyu Chen, Dongyun Zou, Wenkun He, Junsong Chen, Enze Xie, Song Han, and Han Cai. DC-AE 1.5 : Accelerating diffusion model convergence with structured latent space. In IEEE/CVF International Conference on Computer Vision (ICCV), 2025 b

2025
[185]

WavLM : Large-scale self-supervised pre-training for full stack speech processing

Sanyuan Chen, Chengyi Wang, Zhengyang Chen, Yu Wu, Shujie Liu, Zhuo Chen, Jinyu Li, Naoyuki Kanda, Takuya Yoshioka, Xiong Xiao, et al. WavLM : Large-scale self-supervised pre-training for full stack speech processing. IEEE Journal of Selected Topics in Signal Processing, 2022

2022
[186]

cudnn: Efficient primitives for deep learning

Sharan Chetlur, Cliff Woolley, Philippe Vandermersch, Jonathan Cohen, John Tran, Bryan Catanzaro, and Evan Shelhamer. cudnn: Efficient primitives for deep learning. CoRR, abs/1410.0759, 2014. http://arxiv.org/abs/1410.0759

2014 arXiv
[187]

Vision transformers need registers

Timoth \'e e Darcet, Maxime Oquab, Julien Mairal, and Piotr Bojanowski. Vision transformers need registers. In International Conference on Learning Representations (ICLR), 2024

2024
[188]

Oasis : A universe in a transformer, 2024

Decart and Etched . Oasis : A universe in a transformer, 2024. https://oasis-model.github.io/

2024
[189]

Moshi : a speech-text foundation model for real-time dialogue

Alexandre D \'e fossez, Laurent Mazar \'e , Manu Orsini, Am \'e lie Royer, Patrick P \'e rez, Herv \'e J \'e gou, Edouard Grave, and Neil Zeghidour. Moshi : a speech-text foundation model for real-time dialogue. arXiv preprint arXiv:2410.00037, 2024

2024 arXiv
[190]

Scaling vision transformers to 22 billion parameters

Mostafa Dehghani, Josip Djolonga, Basil Mustafa, Piotr Padlewski, Jonathan Heek, et al. Scaling vision transformers to 22 billion parameters. In International Conference on Machine Learning (ICML), 2023

2023
[191]

An image is worth 16x16 words: Transformers for image recognition at scale

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at...

2021
[192]

WorldScore : A unified evaluation benchmark for world generation

Haoyi Duan, Hong-Xing Guo, Xiaoshuai Zhao, Jiajun Wu, et al. WorldScore : A unified evaluation benchmark for world generation. In IEEE/CVF International Conference on Computer Vision (ICCV), 2025

2025
[193]

Taming transformers for high-resolution image synthesis

Patrick Esser, Robin Rombach, and Bj \"o rn Ommer. Taming transformers for high-resolution image synthesis. In CVPR, 2021

2021
[194]

The Matrix : Infinite-horizon world generation with real-time moving control

Ruili Feng, Han Zhang, Zhantao Yang, Jie Xiao, Zhilei Shu, Zhiheng Liu, Andy Zheng, Yukun Huang, Yu Liu, and Hongyang Zhang. The Matrix : Infinite-horizon world generation with real-time moving control. arXiv preprint arXiv:2412.03568, 2024

2024 arXiv
[195]

One step diffusion via shortcut models

Kevin Frans, Danijar Hafner, Sergey Levine, and Pieter Abbeel. One step diffusion via shortcut models. In International Conference on Learning Representations (ICLR), 2025

2025
[196]

Vista : A generalizable driving world model with high fidelity and versatile controllability

Shenyuan Gao, Jiazhi Yang, Li Chen, Kashyap Chitta, Yihang Qiu, Andreas Geiger, Jun Zhang, and Hongyang Li. Vista : A generalizable driving world model with high fidelity and versatile controllability. In Advances in Neural Information Processing Systems (NeurIPS), 2024

2024
[197]

One layer is enough: Adapting pretrained visual encoders for image generation

Yuan Gao, Chen Chen, Tianrong Chen, and Jiatao Gu. One layer is enough: Adapting pretrained visual encoders for image generation. arXiv preprint arXiv:2512.07829, 2025

2025
[198]

Evaluating Gemini robotics policies in a Veo world simulator

Gemini Robotics Team . Evaluating Gemini robotics policies in a Veo world simulator. 2025

2025
[199]

Genie 2 : A large-scale foundation world model, 2024

Google DeepMind . Genie 2 : A large-scale foundation world model, 2024. https://deepmind.google/blog/genie-2-a-large-scale-foundation-world-model/

2024
[200]

Genie 3 : A new frontier for world models, 2025

Google DeepMind . Genie 3 : A new frontier for world models, 2025. https://deepmind.google/blog/genie-3-a-new-frontier-for-world-models/

2025
[201]

Adapting self-supervised representations as a latent space for efficient generation

Ming Gui, Johannes Schusterbauer, Timy Phan, Felix Krause, Josh Susskind, Miguel Angel Bautista, and Bj \"o rn Ommer. Adapting self-supervised representations as a latent space for efficient generation. arXiv preprint arXiv:2510.14630, 2025

2025 arXiv
[202]

MineWorld : A real-time and open-source interactive world model on Minecraft

Junliang Guo, Yang Ye, Tianyu He, Haoyu Wu, Yushu Jiang, Tim Pearce, and Jiang Bian. MineWorld : A real-time and open-source interactive world model on Minecraft . arXiv preprint arXiv:2504.08388, 2025

2025 arXiv
[203]

Photorealistic video generation with diffusion models

Agrim Gupta, Lijun Yu, Kihyuk Sohn, Xiuye Gu, Meera Hahn, Li Fei-Fei, Irfan Essa, Lu Jiang, and Jos \'e Lezama. Photorealistic video generation with diffusion models. In European Conference on Computer Vision (ECCV), 2024

2024
[204]

Recurrent world models facilitate policy evolution

David Ha and J \"u rgen Schmidhuber. Recurrent world models facilitate policy evolution. In Advances in Neural Information Processing Systems (NeurIPS), 2018. arXiv:1803.10122

2018 arXiv
[205]

LTX-Video : Realtime video latent diffusion

Yoav HaCohen et al. LTX-Video : Realtime video latent diffusion. arXiv preprint arXiv:2501.00103, 2025

2025 arXiv
[206]

LTX-2 : Efficient joint audio-visual foundation model

Yoav HaCohen et al. LTX-2 : Efficient joint audio-visual foundation model. arXiv preprint arXiv:2601.03233, 2026

2026 arXiv
[207]

Learning latent dynamics for planning from pixels

Danijar Hafner, Timothy Lillicrap, Ian Fischer, Ruben Villegas, David Ha, Honglak Lee, and James Davidson. Learning latent dynamics for planning from pixels. In International Conference on Machine Learning (ICML), 2019

2019
[208]

Dream to control: Learning behaviors by latent imagination

Danijar Hafner, Timothy Lillicrap, Jimmy Ba, and Mohammad Norouzi. Dream to control: Learning behaviors by latent imagination. In International Conference on Learning Representations (ICLR), 2020

2020
[209]

Mastering atari with discrete world models

Danijar Hafner, Timothy Lillicrap, Mohammad Norouzi, and Jimmy Ba. Mastering atari with discrete world models. In International Conference on Learning Representations (ICLR), 2021

2021
[210]

Mastering diverse domains through world models

Danijar Hafner, Jurgis Pasukonis, Jimmy Ba, and Timothy Lillicrap. Mastering diverse domains through world models. Nature, 2025 a . arXiv:2301.04104

2025 arXiv
[211]

Mastering diverse control tasks through world models

Danijar Hafner, Jurgis Pasukonis, Jimmy Ba, and Timothy Lillicrap. Mastering diverse control tasks through world models. Nature, 2025 b

2025
[212]

Training agents inside of scalable world models

Danijar Hafner, Wilson Yan, and Timothy Lillicrap. Training agents inside of scalable world models. arXiv preprint arXiv:2509.24527, 2025 c

2025 arXiv
[213]

Temporal difference learning for model predictive control

Nicklas Hansen, Xiaolong Wang, and Hao Su. Temporal difference learning for model predictive control. In International Conference on Machine Learning (ICML), 2022

2022
[214]

TD-MPC2 : Scalable, robust world models for continuous control

Nicklas Hansen, Hao Su, and Xiaolong Wang. TD-MPC2 : Scalable, robust world models for continuous control. In International Conference on Learning Representations (ICLR), 2024

2024
[215]

ViTok-v2 : Scaling native resolution auto-encoders to 5 billion parameters

Philippe Hansen-Estruch, Jiahui Chen, Vivek Ramanujan, Orr Zohar, Yan Ping, Animesh Sinha, Markos Georgopoulos, Edgar Schoenfeld, Ji Hou, Felix Juefei-Xu, Sriram Vishwanath, and Ali Thabet. ViTok-v2 : Scaling native resolution auto-encoders to 5 billion parameters. arXiv prepr...

2026 arXiv
[216]

CameraCtrl : Enabling camera control for text-to-video generation

Hao He, Yinghao Xu, Yuwei Guo, Gordon Wetzstein, Bo Dai, Hongsheng Li, and Ceyuan Yang. CameraCtrl : Enabling camera control for text-to-video generation. arXiv preprint arXiv:2404.02101, 2024

2024 arXiv
[217]

Masked autoencoders are scalable vision learners

Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Doll \'a r, and Ross Girshick. Masked autoencoders are scalable vision learners. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022

2022
[218]

Matrix-Game 2.0 : An open-source, real-time, and streaming interactive world model

Xianglong He, Chunli Zhang, Dongdong Wu, Yifan Zhang, Yiqun Xu, et al. Matrix-Game 2.0 : An open-source, real-time, and streaming interactive world model. arXiv preprint arXiv:2508.13009, 2025

2025 arXiv
[219]

Query-key normalization for transformers

Alex Henry, Prudhvi Raj Dachapally, Shubham Pawar, and Yuxuan Chen. Query-key normalization for transformers. In Findings of the Association for Computational Linguistics: EMNLP, 2020

2020
[220]

GANs trained by a two time-scale update rule converge to a local Nash equilibrium

Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. GANs trained by a two time-scale update rule converge to a local Nash equilibrium. In Advances in Neural Information Processing Systems (NeurIPS), 2017

2017
[221]

Denoising diffusion probabilistic models

Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems (NeurIPS), 2020

2020
[222]

Kingma, Ben Poole, Mohammad Norouzi, David J

Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P. Kingma, Ben Poole, Mohammad Norouzi, David J. Fleet, and Tim Salimans. Imagen Video : High definition video generation with diffusion models. arXiv preprint arXiv:2210.02303, 2022 a

2022 arXiv
[223]

Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J. Fleet. Video diffusion models. In Advances in Neural Information Processing Systems (NeurIPS), 2022 b

2022
[224]

Model-based imitation learning for urban driving

Anthony Hu, Gianluca Corrado, Nicolas Griffiths, Zak Murez, Corina Gurau, Hudson Yeo, Alex Kendall, Roberto Cipolla, and Jamie Shotton. Model-based imitation learning for urban driving. In Advances in Neural Information Processing Systems (NeurIPS), 2022

2022
[225]

GAIA-1 : A generative world model for autonomous driving

Anthony Hu, Lloyd Russell, Hudson Yeo, Zak Murez, George Fedoseev, Alex Kendall, Jamie Shotton, and Gianluca Corrado. GAIA-1 : A generative world model for autonomous driving. arXiv preprint arXiv:2309.17080, 2023

2023 arXiv
[226]

MetaWorld : Scaling multi-agent video world model from single-view video data

Teng Hu et al. MetaWorld : Scaling multi-agent video world model from single-view video data. arXiv preprint arXiv:2606.02753, 2026

2026 arXiv
[227]

Self forcing: Bridging the train-test gap in autoregressive video diffusion

Xun Huang, Zhengqi Li, Guande He, Mingyuan Zhou, and Eli Shechtman. Self forcing: Bridging the train-test gap in autoregressive video diffusion. In Advances in Neural Information Processing Systems (NeurIPS), 2025

2025
[228]

VBench : Comprehensive benchmark suite for video generative models

Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, et al. VBench : Comprehensive benchmark suite for video generative models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (...

2024
[229]

World and human action models towards gameplay ideation

Anssi Kanervisto, Dave Bignell, Linda Yilin Wen, Martin Grayson, Raluca Georgescu, Sergio Valcarcel Macua, Shan Zheng Tan, Tabish Rashid, Tim Pearce, Cristian Cant \'o n Ferrer, et al. World and human action models towards gameplay ideation. Nature, 638: 0 656--663, 2025

2025
[230]

FIFO-Diffusion : Generating infinite videos from text without training

Jihwan Kim, Junoh Kang, Jinyoung Choi, and Bohyung Han. FIFO-Diffusion : Generating infinite videos from text without training. In Advances in Neural Information Processing Systems (NeurIPS), 2024

2024
[231]

Learning to simulate dynamic environments with GameGAN

Seung Wook Kim, Yuhao Zhou, Jonah Philion, Antonio Torralba, and Sanja Fidler. Learning to simulate dynamic environments with GameGAN . In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020

2020
[232]

Pathdreamer : A world model for indoor navigation

Jing Yu Koh, Honglak Lee, Yinfei Yang, Jason Baldridge, and Peter Anderson. Pathdreamer : A world model for indoor navigation. In IEEE/CVF International Conference on Computer Vision (ICCV), 2021

2021
[233]

VideoPoet : A large language model for zero-shot video generation

Dan Kondratyuk, Lijun Yu, Xiuye Gu, Jos \'e Lezama, Jonathan Huang, Grant Schindler, Rachel Hornung, Vighnesh Birodkar, Jimmy Yan, Ming-Chang Chiu, et al. VideoPoet : A large language model for zero-shot video generation. In International Conference on Machine Learning (ICML), 2024

2024
[234]

HunyuanVideo : A systematic framework for large video generative models

Weijie Kong et al. HunyuanVideo : A systematic framework for large video generative models. arXiv preprint arXiv:2412.03603, 2024

2024 arXiv
[235]

EQ-VAE : Equivariance regularized latent space for improved generative image modeling

Theodoros Kouzelis, Ioannis Kakogeorgiou, Spyros Gidaris, and Nikos Komodakis. EQ-VAE : Equivariance regularized latent space for improved generative image modeling. In International Conference on Machine Learning (ICML), 2025

2025
[236]

Professor forcing: A new algorithm for training recurrent networks

Alex Lamb, Anirudh Goyal, Ying Zhang, Saizheng Zhang, Aaron Courville, and Yoshua Bengio. Professor forcing: A new algorithm for training recurrent networks. In Advances in Neural Information Processing Systems (NeurIPS), 2016

2016
[237]

A path towards autonomous machine intelligence, 2022

Yann LeCun. A path towards autonomous machine intelligence, 2022. Position paper, version 0.9.2, OpenReview

2022
[238]

WorldModelBench : Judging video generation models as world models

Dacheng Li, Yunhao Zhang, Ji Lin, Enze Xie, et al. WorldModelBench : Judging video generation models as world models. arXiv preprint arXiv:2502.20694, 2025 a

2025 arXiv
[239]

Hunyuan-GameCraft : High-dynamic interactive game video generation with hybrid history condition

Jiaqi Li, Junshu Zhang, Boyuan Jiang, Yuxuan Wang, Yujie Zhao, et al. Hunyuan-GameCraft : High-dynamic interactive game video generation with hybrid history condition. arXiv preprint arXiv:2506.17201, 2025 b

2025 arXiv
[240]

Hopkins, David Bau, Fernanda Vi \'e gas, Hanspeter Pfister, and Martin Wattenberg

Kenneth Li, Aspen K. Hopkins, David Bau, Fernanda Vi \'e gas, Hanspeter Pfister, and Martin Wattenberg. Emergent world representations: Exploring a sequence model trained on a synthetic task. In International Conference on Learning Representations (ICLR), 2023

2023
[241]

Back to basics: Let denoising generative models denoise

Tianhong Li and Kaiming He. Back to basics: Let denoising generative models denoise. arXiv preprint arXiv:2511.13720, 2025

2025 arXiv
[242]

InfiniteNature-Zero : Learning perpetual view generation of natural scenes from single images

Zhengqi Li, Qianqian Wang, Noah Snavely, and Angjoo Kanazawa. InfiniteNature-Zero : Learning perpetual view generation of natural scenes from single images. In European Conference on Computer Vision (ECCV), 2022

2022
[243]

Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. In International Conference on Learning Representations (ICLR), 2023

2023
[244]

Infinite Nature : Perpetual view generation of natural scenes from a single image

Andrew Liu, Richard Tucker, Varun Jampani, Ameesh Makadia, Noah Snavely, and Angjoo Kanazawa. Infinite Nature : Perpetual view generation of natural scenes from a single image. In IEEE/CVF International Conference on Computer Vision (ICCV), 2021

2021
[245]

-World : Generative multi-agent world modeling beyond two players

Fangfu Liu, Kai He, Xuanchi Ren, et al. -World : Generative multi-agent world modeling beyond two players. arXiv preprint arXiv:2605.28816, 2026

2026 arXiv
[246]

Rolling forcing: Autoregressive long video diffusion in real time

Kunhao Liu, Wenbo Li, Jiale Zhao, Ziwei Liu, Shijian Lu, et al. Rolling forcing: Autoregressive long video diffusion in real time. arXiv preprint arXiv:2509.25161, 2025

2025 arXiv
[247]

Zero-1-to-3 : Zero-shot one image to 3d object

Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tokmakov, Sergey Zakharov, and Carl Vondrick. Zero-1-to-3 : Zero-shot one image to 3d object. In IEEE/CVF International Conference on Computer Vision (ICCV), 2023 a

2023
[248]

Flow straight and fast: Learning to generate and transfer data with rectified flow

Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. In International Conference on Learning Representations (ICLR), 2023 b

2023
[249]

Latent consistency models: Synthesizing high-resolution images with few-step inference

Simian Luo, Yiqin Tan, Longbo Huang, Jian Li, and Hongsheng Li. Latent consistency models: Synthesizing high-resolution images with few-step inference. arXiv preprint arXiv:2310.04378, 2023

2023 arXiv
[250]

Albergo, Nicholas M

Nanye Ma, Mark Goldstein, Michael S. Albergo, Nicholas M. Boffi, Eric Vanden-Eijnden, and Saining Xie. SiT : Exploring flow and diffusion-based generative models with scalable interpolant transformers. In European Conference on Computer Vision (ECCV), 2024

2024
[251]

PixelGen : Improving pixel diffusion with perceptual supervision

Zehong Ma et al. PixelGen : Improving pixel diffusion with perceptual supervision. arXiv preprint arXiv:2602.02493, 2026

2026 arXiv
[252]

LeWorldModel : Stable end-to-end joint-embedding predictive architecture from pixels

Lucas Maes, Quentin Le Lidec, Damien Scieur, Yann LeCun, and Randall Balestriero. LeWorldModel : Stable end-to-end joint-embedding predictive architecture from pixels. arXiv preprint arXiv:2603.19312, 2026

2026 arXiv
[253]

Medal : A gameplay clip capture and sharing platform

Medal.tv . Medal : A gameplay clip capture and sharing platform. https://medal.tv, 2024

2024
[254]

Playable video generation

Willi Menapace, St \'e phane Lathuili \`e re, Sergey Tulyakov, Aliaksandr Siarohin, and Elisa Ricci. Playable video generation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021

2021
[255]

Towards world simulator: Crafting physical commonsense-based benchmark for video generation

Fanqing Meng, Jiaqi Liao, Xinyu Tan, Wenqi Shao, Quanfeng Lu, Kaipeng Zhang, Yu Cheng, Dianqi Li, Yu Qiao, and Ping Luo. Towards world simulator: Crafting physical commonsense-based benchmark for video generation. In International Conference on Machine Learning (ICML), 2025

2025
[256]

EUPE : Efficient universal perception encoder

Meta AI . EUPE : Efficient universal perception encoder. arXiv preprint arXiv:2603.22387, 2026

2026
[257]

Transformers are sample-efficient world models

Vincent Micheli, Eloi Alonso, and Fran c ois Fleuret. Transformers are sample-efficient world models. In International Conference on Learning Representations (ICLR), 2023

2023
[258]

Efficient world models with context-aware tokenization

Vincent Micheli, Eloi Alonso, and Fran c ois Fleuret. Efficient world models with context-aware tokenization. In International Conference on Machine Learning (ICML), 2024

2024
[259]

Do generative video models understand physical principles? arXiv preprint arXiv:2501.09038, 2025

Saman Motamed, Laura Culp, Kevin Swersky, Priyank Jaini, and Robert Geirhos. Do generative video models understand physical principles? arXiv preprint arXiv:2501.09038, 2025

2025 arXiv
[260]

BakkesMod : A rocket league modding framework

Chris Mulder and BakkesMod contributors . BakkesMod : A rocket league modding framework. https://bakkesmod.com, 2016

2016
[261]

DINOv2 : Learning robust visual features without supervision

Maxime Oquab, Timoth \'e e Darcet, Th \'e o Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. DINOv2 : Learning robust visual features without supervision. Transactions on Machine Learning Research...

2024
[262]

Scalable diffusion models with transformers

William Peebles and Saining Xie. Scalable diffusion models with transformers. In IEEE/CVF International Conference on Computer Vision (ICCV), 2023

2023
[263]

BEiT v2 : Masked image modeling with vector-quantized visual tokenizers

Zhiliang Peng, Li Dong, Hangbo Bao, Qixiang Ye, and Furu Wei. BEiT v2 : Masked image modeling with vector-quantized visual tokenizers. arXiv preprint arXiv:2208.06366, 2022

2022 arXiv
[264]

MultiGen : Level-design for editable multiplayer worlds in diffusion game engines

Ryan Po, Kai Zhang, Amir Hertz, Gordon Wetzstein, Neal Wadhwa, and Nataniel Ruiz. MultiGen : Level-design for editable multiplayer worlds in diffusion game engines. arXiv preprint arXiv:2603.06679, 2026

2026
[265]

Movie Gen : A cast of media foundation models

Adam Polyak et al. Movie Gen : A cast of media foundation models. arXiv preprint arXiv:2410.13720, 2024

2024 arXiv
[266]

ActionParty : Multi-subject action binding in generative video games

Alexander Pondaven, Haoyu Wu, Igor Gilitschenski, Philip Torr, Sergey Tulyakov, Fabio Pizzati, and Aliaksandr Siarohin. ActionParty : Multi-subject action binding in generative video games. arXiv preprint arXiv:2604.02330, 2026

2026
[267]

Gated attention for large language models: Non-linearity, sparsity, and attention-sink-free

Zihan Qiu et al. Gated attention for large language models: Non-linearity, sparsity, and attention-sink-free. In Advances in Neural Information Processing Systems (NeurIPS), 2025

2025
[268]

Worldgym: World model as an environment for policy evaluation, 2025

Julian Quevedo, Ansh Kumar Sharma, Yixiang Sun, Varad Suryavanshi, Percy Liang, and Sherry Yang. Worldgym: World model as an environment for policy evaluation, 2025

2025
[269]

One View Is Enough ! monocular training for in-the-wild novel view generation

Adrien Ramanana Rahary, Nicolas Dufour, Patrick P \'e rez, and David Picard. One View Is Enough ! monocular training for in-the-wild novel view generation. arXiv preprint arXiv:2603.23488, 2026

2026 arXiv
[270]

RLGym : A python api for reinforcement learning in rocket league

RLGym contributors . RLGym : A python api for reinforcement learning in rocket league. https://rlgym.org, 2021

2021
[271]

Transformer-based world models are happy with 100k interactions

Jan Robine, Marc H \"o ftmann, Tobias Uelwer, and Stefan Harmeling. Transformer-based world models are happy with 100k interactions. In International Conference on Learning Representations (ICLR), 2023

2023
[272]

High-resolution image synthesis with latent diffusion models

Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj \"o rn Ommer. High-resolution image synthesis with latent diffusion models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022

2022
[273]

Rolling diffusion models

David Ruhe, Jonathan Heek, Tim Salimans, and Emiel Hoogeboom. Rolling diffusion models. In International Conference on Machine Learning (ICML), 2024

2024
[274]

GAIA-2 : A controllable multi-view generative world model for autonomous driving

Lloyd Russell, Anthony Hu, Lorenzo Bertoni, George Fedoseev, Jamie Shotton, Elahe Arani, and Gianluca Corrado. GAIA-2 : A controllable multi-view generative world model for autonomous driving. arXiv preprint arXiv:2503.20523, 2025

2025 arXiv
[275]

Improved techniques for training GANs

Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training GANs . In Advances in Neural Information Processing Systems (NeurIPS), 2016

2016
[276]

ZeroNVS : Zero-shot 360-degree view synthesis from a single image

Kyle Sargent, Zizhang Li, Tanmay Shah, Charles Herrmann, Hong-Xing Yu, Yunzhi Zhang, Eric Ryan Chan, Dmitry Lagun, Li Fei-Fei, Deqing Sun, and Jiajun Wu. ZeroNVS : Zero-shot 360-degree view synthesis from a single image. In IEEE/CVF Conference on Computer Vision and Pattern Re...

2024
[277]

Solaris : Building a multiplayer video world model in Minecraft

Georgy Savva, Oscar Michel, Saining Xie, et al. Solaris : Building a multiplayer video world model in Minecraft . arXiv preprint arXiv:2602.22208, 2026

2026
[278]

Mastering Atari , Go , chess and shogi by planning with a learned model

Julian Schrittwieser, Ioannis Antonoglou, Thomas Hubert, Karen Simonyan, Laurent Sifre, Simon Schmitt, Arthur Guez, Edward Lockhart, Demis Hassabis, Thore Graepel, Timothy Lillicrap, and David Silver. Mastering Atari , Go , chess and shogi by planning with a learned model. Nat...

2020
[279]

Fast transformer decoding: One write-head is all you need

Noam Shazeer. Fast transformer decoding: One write-head is all you need. arXiv preprint arXiv:1911.02150, 2019

1911 arXiv
[280]

GLU variants improve transformer

Noam Shazeer. GLU variants improve transformer. arXiv preprint arXiv:2002.05202, 2020

2002 arXiv
[281]

Welcome to the era of experience

David Silver and Richard S Sutton. Welcome to the era of experience. Google AI, 2025

2025
[282]

Vo, Maximilian Seitzer, Federico Baldassarre, Maxime Oquab, Cijo Jose, Vasil Khalidov, Marc Szafraniec, Seungeun Yi, Micha \"e l Ramamonjisoa, et al

Oriane Sim \'e oni, Huy V. Vo, Maximilian Seitzer, Federico Baldassarre, Maxime Oquab, Cijo Jose, Vasil Khalidov, Marc Szafraniec, Seungeun Yi, Micha \"e l Ramamonjisoa, et al. DINOv3 . arXiv preprint arXiv:2508.10104, 2025

2025 arXiv
[283]

Make-A-Video : Text-to-video generation without text-video data

Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, Devi Parikh, Sonal Gupta, and Yaniv Taigman. Make-A-Video : Text-to-video generation without text-video data. In International Conference on Learning Repres...

2023
[284]

Improved baselines with representation autoencoders

Jaskirat Singh, Boyang Zheng, Zongze Wu, Richard Zhang, Eli Shechtman, and Saining Xie. Improved baselines with representation autoencoders. arXiv preprint arXiv:2605.18324, 2026

2026 arXiv
[285]

History-guided video diffusion

Kiwhan Song, Boyuan Chen, Max Simchowitz, Yilun Du, Russ Tedrake, and Vincent Sitzmann. History-guided video diffusion. In International Conference on Machine Learning (ICML), 2025

2025
[286]

Improved techniques for training consistency models

Yang Song and Prafulla Dhariwal. Improved techniques for training consistency models. In International Conference on Learning Representations (ICLR), 2024

2024
[287]

Consistency models

Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. In International Conference on Machine Learning (ICML), 2023

2023
[288]

RoFormer : Enhanced transformer with rotary position embedding

Jianlin Su, Yu Lu, Shengfeng Pan, Ahmed Murtadha, Bo Wen, and Yunfeng Liu. RoFormer : Enhanced transformer with rotary position embedding. Neurocomputing, 2024

2024
[289]

Philippe Tillet, H. T. Kung, and David Cox. Triton: an intermediate language and compiler for tiled neural network computations. In Proceedings of the 3rd ACM SIGPLAN International Workshop on Machine Learning and Programming Languages, MAPL 2019, page 10–19, New York, NY, USA...

2019 doi
[290]

Scaling text-to-image diffusion transformers with representation autoencoders

Shengbang Tong, Boyang Zheng, Ziteng Wang, Bingda Tang, Nanye Ma, Ellis Brown, Jihan Yang, Rob Fergus, Yann LeCun, and Saining Xie. Scaling text-to-image diffusion transformers with representation autoencoders. arXiv preprint arXiv:2601.16208, 2026

2026
[291]

SigLIP 2 : Multilingual vision-language encoders with improved semantic understanding, localization, and dense features

Michael Tschannen, Alexey Gritsenko, Xiao Wang, Muhammad Ferjad Naeem, Ibrahim Alabdulmohsin, et al. SigLIP 2 : Multilingual vision-language encoders with improved semantic understanding, localization, and dense features. arXiv preprint arXiv:2502.14786, 2025

2025 arXiv
[292]

Towards accurate generative models of video: A new metric and challenges

Thomas Unterthiner, Sjoerd van Steenkiste, Karol Kurach, Raphael Marinier, Marcin Michalski, and Sylvain Gelly. Towards accurate generative models of video: A new metric and challenges. arXiv preprint arXiv:1812.01717, 2018

2018 arXiv
[293]

Diffusion models are real-time game engines

Dani Valevski, Yaniv Leviathan, Moab Arar, and Shlomi Fruchter. Diffusion models are real-time game engines. In International Conference on Learning Representations (ICLR), 2025

2025
[294]

Worldgen: From text to traversable and interactive 3d worlds

Dilin Wang, Hyunyoung Jung, Tom Monnier, Kihyuk Sohn, Chuhang Zou, Xiaoyu Xiang, Yu-Ying Yeh, Di Liu, Zixuan Huang, Thu Nguyen-Phuoc, et al. Worldgen: From text to traversable and interactive 3d worlds. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2026

2026
[295]

DriveDreamer : Towards real-world-driven world models for autonomous driving

Xiaofeng Wang, Zheng Zhu, Guan Huang, Xinze Chen, Jiagang Zhu, and Jiwen Lu. DriveDreamer : Towards real-world-driven world models for autonomous driving. In European Conference on Computer Vision (ECCV), 2024

2024
[296]

Bovik, Hamid R

Zhou Wang, Alan C. Bovik, Hamid R. Sheikh, and Eero P. Simoncelli. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13 0 (4), 2004

2004
[297]

GAIA-3 : Advancing world models from simulation to evaluation, 2025

Wayve . GAIA-3 : Advancing world models from simulation to evaluation, 2025. https://wayve.ai/press/wayve-launches-gaia3/

2025
[298]

Generating worlds, 2024

World Labs . Generating worlds, 2024. https://www.worldlabs.ai/blog/generating-worlds

2024
[299]

RTFM : A real-time frame model, 2025

World Labs . RTFM : A real-time frame model, 2025. https://www.worldlabs.ai/blog/rtfm

2025
[300]

MultiWorld : Scalable multi-agent multi-view video world models

Haoyu Wu, Jiwen Yu, Yingtian Zou, and Xihui Liu. MultiWorld : Scalable multi-agent multi-view video world models. arXiv preprint arXiv:2604.18564, 2026

2026 arXiv

Pith tools

Reviewed July 7, 2026 · model on record in the stance chip above.

[1] [1]

Advances in Neural Information Processing Systems (NeurIPS) , year =

Recurrent World Models Facilitate Policy Evolution , author =. Advances in Neural Information Processing Systems (NeurIPS) , year =

work page

[2] [2]

International Conference on Machine Learning (ICML) , year =

Learning Latent Dynamics for Planning from Pixels , author =. International Conference on Machine Learning (ICML) , year =

work page

[3] [3]

International Conference on Learning Representations (ICLR) , year =

Dream to Control: Learning Behaviors by Latent Imagination , author =. International Conference on Learning Representations (ICLR) , year =

work page

[4] [4]

International Conference on Learning Representations (ICLR) , year =

Mastering Atari with Discrete World Models , author =. International Conference on Learning Representations (ICLR) , year =

work page

[5] [5]

Nature , year =

Mastering Diverse Domains through World Models , author =. Nature , year =

work page

[6] [7]

International Conference on Learning Representations (ICLR) , year =

Transformers are Sample-Efficient World Models , author =. International Conference on Learning Representations (ICLR) , year =

work page

[7] [8]

International Conference on Learning Representations (ICLR) , year =

Transformer-based World Models Are Happy With 100k Interactions , author =. International Conference on Learning Representations (ICLR) , year =

work page

[8] [9]

Zhang, Weipu and Wang, Gang and Sun, Jian and Yuan, Yetian and Huang, Gao , booktitle =

work page