{"id":"bc340486-b8c7-40cf-972b-6a49b2669d1c","arxiv_id":"2607.05352","paper_version":1,"verdict":"ACCEPT","confidence":"UNKNOWN","novelty_score":7.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":9,"one_line_summary":"A 5B-parameter latent diffusion model generates real-time four-player Rocket League matches conditioned on all players' actions, staying stable far beyond its training horizon.","lead":"MIRA is a 5-billion-parameter latent diffusion world model that simulates four-player Rocket League matches in real time at 20 fps, conditioned on all four players' actions. It demonstrates that multi-agent conditioning, a frozen pretrained feature extractor for the latent space, and diffusion forcing together yield stable, controllable long-horizon rollouts.","discovery_kind":"unclear","skeptic_critique":{"model":"glm-5.2","headline":"Long-horizon dynamical fidelity is asserted but only distributional metrics (gFID) are tracked over time; the physics probe and ARR are not measured as a function of rollout horizon, leaving open whether visual plausibility masks dynamical drift.","rationale":"The reader correctly identified the single-bot-policy concern, which is real but acknowledged by the authors and partially mitigated by the live demo. My concern is different and more specific: the evaluation architecture creates a gap between what is measured over long horizons (distributional visual quality via gFID) and what measures dynamical fidelity (game-state probe, ARR), with no metric tracking the latter over time. This is a genuine evaluation gap that directly bears on the claim of long-horizon dynamical faithfulness. However, I recommend UNCHANGED because: (1) the paper's primary contribution — a working real-time multiplayer world model — is well-supported by the live demo, code release, and extensive ablations; (2) the gap is one of measurement completeness rather than evidence of failure; (3) the qualitative evidence (Figures 5, 6, 21) and the live demo provide reasonable if non-quantitative support for dynamical fidelity; (4) the concern narrows the scope of the long-horizon claim but does not undermine the core technical contribution. The paper is honest about its limitations and the claim is appropriately hedged in places. An ACCEPT with the caveat that long-horizon dynamical fidelity is not directly measured remains defensible.","tokens_in":50236,"tokens_out":2898,"duration_ms":195434,"concrete_test":"Measure game-state probe error (L2 distance for ball and car positions/velocities) as a function of rollout horizon, at the same time points where gFID is already measured (1s, 4s, 12s, 30s, 60s, 120s, 300s), on the 5B model with ground-truth actions. If probe error grows substantially over the rollout while gFID stays flat, the claim of long-horizon dynamical fidelity weakens; if both stay flat, the claim is well-supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim states that rollouts are 'both visually coherent and dynamically faithful to the commanded actions' and stay stable 'far beyond the training horizon' (measured to 5 minutes). The paper supports long-horizon stability with gFID/gFVD/gFDD tracked over rollout time (Figures 9, 7), and supports dynamical fidelity with the game-state probe (Figure 21, Figure 16a) and ARR (Figure 13). However, these two lines of evidence are measured at different horizons: the distributional metrics run out to 300 seconds, while the dynamical metrics do not. The game-state probe error (Figure 16a) is reported only as a function of model size, not rollout horizon — the reader cannot tell whether physics accuracy at 4 seconds differs from physics accuracy at 60 or 300 seconds. ARR (Figure 13) is reported over training steps, not over rollout time. This matters because gFID measures whether generated frames look realistic on average, not whether any individual rollout follows correct physics. A model could maintain flat gFID over 5 minutes while the ball drifts through walls or cars teleport, as long as each frame is individually plausible. The qualitative probe overlay in Figure 21 is shown for a short rollout only. The claim of dynamical fidelity over long horizons therefore rests on an extrapolation: short-horizon dynamical metrics plus long-horizon distributional metrics, with no measurement bridging the two. This is the soft spot — not because the model necessarily fails, but because the evaluation does not directly test the combined claim (long-horizon AND dynamically faithful).","agreement_with_reader":"partial"},"referee_report":{"model":"glm-5.2","summary":"This paper introduces MIRA, a 5B-parameter latent diffusion world model for four-player Rocket League that conditions on all players' simultaneous action streams, generates 20 fps in real time on a single B200 GPU, and remains stable over long horizons. The model predicts in the latent space of a representation autoencoder built on a frozen DINOv3-L feature extractor, trained with flow matching and diffusion forcing. The paper provides systematic ablations of the codec design (Tables 2–7, 21–25), the training objective (Table 8, Figures 9–11), the multiplayer conditioning scheme (Figure 14), and scaling behavior (Figure 16). It also introduces two targeted evaluation metrics: the Action Recoverability Ratio (ARR), validated against human judgment (Pearson r=0.84, Figure 12), and a game-state probe that reads physical quantities from the model's activations. The authors release their dataset, codebase, and a live demo.","tokens_in":50536,"tokens_out":2473,"duration_ms":147614,"significance":"This is a substantial contribution to interactive world modeling. The multiplayer conditioning design—tiling four views into a single grid with per-player action embeddings—is a clean and effective solution to multi-agent attribution. The systematic codec ablations (particularly Table 3 and Figure 7 showing that a frozen pretrained feature extractor is what prevents long-horizon drift, despite worse reconstruction) provide actionable design guidance for the field. The ARR metric and its validation against human preference studies is a valuable methodological contribution. The release of 10,000 hours of gameplay data with aligned physics state, full training/inference code, and a live interactive demo sets a high standard for reproducibility. The real-time inference system (Section 5) is well-engineered and documented.","major_comments":[{"comment":"§6.2, §6.8: The paper claims rollouts are 'both visually coherent and dynamically faithful to the commanded actions' and 'stay stable far beyond the training horizon' (abstract, §1, §6.1). However, the two lines of evidence are measured at different horizons: distributional metrics (gFID/gFVD/gFDD) extend to 300 seconds (Figures 7, 9), while the dynamical metrics—game-state probe error (Figure 16a) and ARR (Figure 13)—are reported only as functions of model size and training step, respectively, not as functions of rollout horizon. The game-state probe overlay in Figure 21 is shown for a short rollout only. This leaves open whether dynamical fidelity degrades over long horizons even as distributional quality remains flat. The claim of long-horizon stability would be substantially strengthened by reporting ARR or probe error at multiple rollout horizons (e.g., 4s, 30s, 60s, 300s). As word,","section":null},{"comment":"§6.8: The 'theory of mind' claim for unconditioned players is presented as an emergent property, but since all training data is generated by a single bot policy (Nexto, §3.1), the model's behavior for dropped-action players may simply reproduce Nexto's policy distribution rather than learning a general agent model. The paper acknowledges this limitation in §3.1 but does not revisit it when making the 'theory of mind' claim in §6.8. The claim should be qualified: the model has learned to imitate the specific training policy for unconditioned players, which is a narrower result than general agent modeling. The live demo with human players (§6.8, Figure 18) provides partial counter-evidence for robustness to distribution shift, but does not directly test whether the unconditioned-player behavior generalizes beyond Nexto's strategy space.","section":null}],"minor_comments":[{"comment":"§4.2: The adaptive gradient-norm balancing rule is described as reusing 'the gradient-norm balancing that VQ-GAN applies to its single adversarial term,' but VQ-GAN balances between reconstruction and discriminator losses, not between two perceptual terms and reconstruction. The analogy could be stated more precisely.","section":null},{"comment":"§6.3, Table 2: The pixel-space ARR is 'calibrated against real frames rather than a reconstruction' because pixel-space models have no codec. This makes the ARR comparison between latent and pixel space not apples-to-apples (the latent ARR divides by APrecon, the pixel ARR divides by APreal). The paper should note this asymmetry explicitly.","section":null},{"comment":"§6.7, Figure 16a: The ball-position probe error is reported in 'Unreal units' but the axis label says '×10³' without specifying the unit in the caption. Adding the unit (uu) to the caption would improve clarity.","section":null},{"comment":"§5: The paper states one full step takes 'roughly 70 ms end to end and produces two video frames (about 35 ms per frame),' which is within the 50 ms budget for 20 fps. It would be useful to report the variance or worst-case latency, since interactive applications are sensitive to tail latency, not just mean throughput.","section":null},{"comment":"§6.9: The failure case of the ball moving on its own when untouched is attributed to data imbalance. It would strengthen the analysis to report how frequently this occurs quantitatively, similar to the uncommanded boost/jump counts.","section":null},{"comment":"Figure 19 caption: The clock drift example shows the clock reading 4:54, 4:53, 4:53, 4:52, 4:54, 4:53 over five seconds. The text says 'it advances far too slowly and even ticks back up,' but the sequence also shows 4:52→4:54, which is a two-second jump forward followed by a one-second jump back. This pattern (both too slow and occasionally jumping) could be described more precisely.","section":null},{"comment":"References: Several cited works have 2026 dates (e.g., Siméoni et al. 2025 for DINOv3 is listed as arXiv:2508.10104, but other references like Tong et al. 2026, Singh et al. 2026, Hansen-Estruch et al. 2026 appear to be from 2026). The mixing of 2025 and 2026 dates should be verified for consistency.","section":null}],"recommendation":"minor_revision","confidential_remarks":"The paper is a strong systems-and-methods contribution with exceptional reproducibility (code, data, live demo). The stress-test concern about the gap between short-horizon dynamical metrics and long-horizon distributional metrics is valid and is the most important issue to address, but it is a missing experiment rather than a load-bearing error—the paper's wording is mostly careful about what is measured where. The 'theory of mind' claim is somewhat overframed given the single-policy training data and should be qualified. I recommend minor revision with the expectation that the authors add at least one dynamical metric (ARR or probe error) measured as a function of rollout horizon, and qualify the theory-of-mind claim."},"author_rebuttal":{"model":"glm-5.2","summary":"We thank the referee for the careful reading and the constructive feedback. Both major comments identify genuine gaps in our evaluation that we will address in revision. Below we respond point by point.","responses":[{"response":"The referee is correct. Our distributional metrics (gFID/gFVD/gFDD) are tracked over rollout horizon up to 300 seconds (Figures 7, 9, 10), but our two dynamical metrics—ARR (Figure 13) and the game-state probe error (Figure 16a)—are reported only as functions of training step and model size, respectively, not as functions of rollout horizon. The game-state probe overlay in Figure 21 is shown for a short rollout only. This is a real gap: it leaves open the possibility that dynamical fidelity degrades over long horizons even as distributional quality remains flat, which would weaken our long-horizon stability claim. We will address this by computing both ARR and the game-state probe error at multiple rollout horizons (4s, 30s, 60s, 120s, 300s) on the flagship 5B model, conditioned on ground-truth actions, and adding the results as a new figure. We will also soften the abstract and §6.1 claims to specify that long-horizon stability has been verified for distributional quality and will be verified for dynamical fidelity in the revision. We note that the game-state probe is trained on real latents and applied to generated rollouts (§6.2), so extending it to multiple horizons is straightforward computationally; the main cost is running the probe over long rollouts, which we can do with existing infrastructure.","revision_made":"yes","referee_comment":"§6.2, §6.8: Distributional metrics extend to 300s but dynamical metrics (ARR, game-state probe) are not reported as functions of rollout horizon. The claim of long-horizon stability would be strengthened by reporting ARR or probe error at multiple rollout horizons (e.g., 4s, 30s, 60s, 300s)."},{"response":"The referee raises a valid concern. We acknowledge in §3.1 that using a single bot policy (Nexto) limits behavioral diversity, but we do not revisit this limitation when making the 'theory of mind' claim in §6.8. The referee is right that, since all four cars in every training match are driven by independent instances of the same Nexto policy, the model's behavior for unconditioned players most plausibly reflects imitation of Nexto's policy distribution learned from pixels, rather than a general agent model that would transfer to arbitrary player strategies. The live demo with human players (§6.8, Figure 18) provides partial counter-evidence—the model stays coherent under human control, which is outside the training action distribution—but this tests robustness to distribution shift in the conditioned player's actions, not whether the unconditioned-player behavior generalizes beyond Nexto's strategy space. We will revise §6.8 to qualify the claim explicitly: the model has learned to imitate the specific training policy for unconditioned players, which is a narrower result than general agent modeling. We will also add a forward reference to the §3.1 limitation at the point where the claim is made. We retain the observation that the model recovers complex decisions from pixels alone (the bot has access to privileged game state), but we will frame this as policy imitation from observation rather than 'theory of mind' in the general sense.","revision_made":"yes","referee_comment":"§6.8: The 'theory of mind' claim for unconditioned players is presented as an emergent property, but since all training data is generated by a single bot policy (Nexto, §3.1), the model's behavior for dropped-action players may simply reproduce Nexto's policy distribution rather than learning a general agent model. The claim should be qualified."}],"tokens_in":49976,"tokens_out":1153,"duration_ms":57286,"standing_objections":[]},"desk_editor":{"model":"glm-5.2","letter":"This paper delivers what it claims: a real-time, four-player, action-conditioned latent diffusion world model for Rocket League, running at 20 fps on a single B200. That is a genuine first. No prior work has conditioned a generative world model on multiple simultaneous action streams in a continuous-physics 3D environment. The systematic ablation work is the real strength — Tables 2–8 isolate each design choice (pixel vs. latent, pretrained vs. from-scratch features, diffusion vs. teacher forcing, codec components) with matched compute and clean controls. The ARR metric is validated against human judgment (Pearson r=0.84), and the game-state probe is a sensible way to test whether the model encodes physics rather than just appearance. Code, data, and a live demo are all released. This is reproducible work at a scale few labs can match, and the authors earn credit for that. The stress-test concern about long-horizon dynamical fidelity is valid but narrower than it sounds. The paper measures gFID out to 300 seconds (Figure 9) and measures the game-state probe and ARR at short horizons. It does not cross the two — no probe error as a function of rollout time. So the claim of dynamical fidelity over long horizons rests on an extrapolation: short-horizon physics accuracy plus long-horizon distributional stability. This is a real gap, but it is a gap in evaluation, not evidence of failure. The flat gFID curves, the live demo under human control, and the qualitative probe overlays all point in the right direction. A referee should ask for the probe error tracked over rollout time, but the central result does not collapse without it. The single-bot-policy training data (Nexto) is the other soft spot. The authors acknowledge it. The emergent “theory of mind” for unconditioned players (Section 6.8) could be Nexto reproduction rather than learned agent modeling, and the paper does not rigorously distinguish the two. This is secondary to the main contribution. This paper is for researchers working on generative world models, video prediction, or learned simulators. It deserves a serious referee. The main asks: (1) game-state probe error as a function of rollout horizon, (2) clearer separation of Nexto reproduction from genuine learned dynamics in the emergent-behavior claims.","headline":"First real-time, four-player, action-conditioned generative world model for a continuous-physics 3D game. The core engineering result is solid and well-supported.","tokens_in":51280,"tokens_out":582,"would_cite":true,"duration_ms":129699,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"glm-5.2","headline":"Four-player world model runs real-time, stays stable for hours","keywords":["world model","multiplayer","latent diffusion","representation autoencoder","diffusion forcing","action conditioning","Rocket League","real-time generation"],"falsifier":"If the model's rollouts degrade significantly when driven by action sequences that differ substantially from the Nexto bot's behavioral distribution — for example, sustained inaction, unusual aerial maneuvers, or adversarial inputs designed to push cars into states the bot never produces — then the model would have demonstrated pattern matching to one policy rather than learning general dynamics.","tokens_in":50375,"feed_emoji":"🎮","tokens_out":1128,"duration_ms":174582,"temperature":0.7,"pith_summary":"The paper claims the first interactive world model that conditions on the simultaneous action streams of four agents in a fast, physically complex environment. The system, called MIRA, is a 5-billion-parameter latent diffusion model trained on 10,000 hours of Rocket League gameplay. It renders all four players' perspectives at 20 frames per second on a single GPU and, despite being trained only on 4-second clips, produces rollouts that remain visually coherent and dynamically faithful for hours. The central mechanism is a combination of three design choices: a latent prediction space built on a frozen pretrained feature extractor (DINOv3), which prevents long-horizon drift; diffusion forcing, which trains the model on partially corrupted context so it can survive its own imperfect predictions at inference; and tiled multiplayer conditioning, which lets a single transformer jointly render all four views and attribute each player's actions to the correct car. The paper argues that these choices together yield a model that captures the game's underlying dynamics rather than merely memorizing visual patterns, supported by evidence that the model stays coherent under human control despite being trained only on bot play.","feed_headline":"Four-player AI world model runs real-time, stays stable for hours","feed_subtitle":"A 5B-parameter latent diffusion model conditions on four agents' actions simultaneously, rendering 20 fps gameplay that holds together far超出","key_machinery":"A representation autoencoder that compresses video into a compact latent by combining a frozen DINOv3 feature extractor with a learned linear bottleneck (2x spatial, 2x temporal downsampling), paired with a flow-matching diffusion transformer that predicts future latents autoregressively. Each frame receives an independent noise level during training (diffusion forcing), and the four players' views are tiled into a single grid so spatial attention can keep them mutually consistent.","core_discovery":"The paper's central result is that conditioning a latent diffusion world model on four simultaneous action streams, predicting in the latent space of a frozen pretrained feature extractor, and training with diffusion forcing yields real-time, multi-agent rollouts that stay stable far beyond the training horizon. The pretrained feature extractor is the load-bearing ingredient for long-horizon stability: codecs built on from-scratch extractors reconstruct video more sharply but drift substantially over sustained rollouts, while the pretrained-extractor latent stays flat. Diffusion forcing is the second pillar, keeping teacher-forced rollouts from collapsing past the 4-second training window. T","pith_inferences":["If the model's stability under human control is genuine (not just interpolation within the bot's behavioral distribution), it would imply the model has learned a generalizable dynamics model rather than a policy imitation, which would be a stronger claim than the paper's single-bot training data would guarantee.","The emergent 'theory of mind' for unconditioned players could be tested more rigorously by training on data from multiple distinct bot policies and checking whether the model can interpolate between their playing styles at inference.","The drift resistance of pretrained feature extractors may connect to the spectral smoothness of self-supervised representations: nearby states mapping to nearby latents would absorb prediction errors rather than amplifying them, a hypothesis that could be tested by measuring the Lipschitz constant of the latent mapping.","The fact that the model generalizes beyond its training action distribution (staying stable when all cars sit still, or under human play) partially addresses the single-policy concern, but a systematic out-of-distribution action evaluation would strengthen the dynamics-learning claim."],"forward_implications":["If the design transfers, agents could be trained or evaluated inside learned simulators for multi-agent tasks without running the real environment, reducing the cost of reinforcement learning.","The finding that pretrained feature extractors prevent rollout drift suggests that the smoothness of the prediction space, not just reconstruction quality, is what makes a latent suitable for autoregressive generation.","The action recoverability ratio (ARR) provides a reusable protocol for measuring whether generative models actually obey control inputs, addressing a gap between visual fidelity and dynamical correctness.","The two-stage training recipe (pretrain single-player, then warm-start multiplayer) offers a practical path for scaling multi-agent conditioning without requiring the full multiplayer compute budget from the start."],"fun_headline_variants":["Pretrained latent keeps four-player world model stable past training horizon","First multiplayer world model generates real-time four-agent Rocket League","Multi-agent latent diffusion runs 20 fps with no collapse beyond five minutes","Pretrained extractor outlasts from-scratch codecs in sustained rollout stability","5B latent diffusion model conditions on four action streams for real-time play"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The model is trained entirely on data from a single bot policy (Nexto) on three fixed maps, so its claims about learning the game's dynamics rest on the assumption that it has captured the underlying physics rather than memorizing one policy's behavioral patterns.","fun_headline_variants_meta":{"raw":{"variants":["Pretrained latent keeps four-player world model stable past training horizon","First multiplayer world model generates real-time four-agent Rocket League","Multi-agent latent diffusion runs 20 fps with no collapse beyond five minutes","Pretrained extractor outlasts from-scratch codecs in sustained rollout stability","5B latent diffusion model conditions on four action streams for real-time play"]},"model":"glm-5.2","effort":"high","cost_usd":0.0,"raw_usage":{"total_tokens":686,"prompt_tokens":596,"completion_tokens":90,"prompt_tokens_details":null},"tokens_in":596,"tokens_out":90,"duration_ms":59152,"temperature":1.0,"reasoning_tokens":null,"cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-07T15:24:37.032067+00:00","model_set":{"reader":"glm-5.2"},"falsifier":"If the model's rollouts degrade significantly when driven by action sequences that differ substantially from the Nexto bot's behavioral distribution — for example, sustained inaction, unusual aerial maneuvers, or adversarial inputs designed to push cars into states the bot never produces — then the model would have demonstrated pattern matching to one policy rather than learning general dynamics.","supporting_citations":[],"review_version":1}