{"id":"a6f8a3ff-4a8b-4cfa-bc5b-b7b425077f59","arxiv_id":"2607.26754","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Coupling explicit state prediction with MoT-style video generation raises mechanics fidelity of Street Fighter 3 rollouts by about 18.6% over stateless game world models.","lead":"StatePlay jointly predicts game internals (health, skill meters, timers) and video frames so generated play follows rules, not just pixels. It matters because current game world models look real yet break win/loss and special-move logic.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"Headline fidelity gain rests on LLM judges that also labeled training data; engine-grounded state checks are underused for the claim.","rationale":"The reader correctly flags the shared Gemini role in dataset construction and mechanics scoring as the weakest assumption behind the strongest claim. I agree that is the single most load-bearing concern: state L1 and MoT/regression ablations are real and internally coherent, but the marketed 18.6% “mechanics fidelity” lift is not yet tied to emulator-enforced rules on generated rollouts. That does not overturn the contribution—explicit state heads clearly help on SF3 relative to a strong in-domain visual baseline—but it keeps the verdict CONDITIONAL rather than a clean accept until engine-backed rule metrics (or multi-game/public artifacts) close the loop. No stronger internal inconsistency (e.g., in the MoT equations or loss design) displaces this evaluation gap.","tokens_in":15583,"tokens_out":648,"duration_ms":13447,"concrete_test":"On the same 100-clip test set, decode each model’s predicted state sequence (or re-extract HUD/memory-aligned HP, meters, timer from generated frames if only pixels are available) and score hard engine rules: (i) super-art only if player meter ≥ τ and meter resets after; (ii) result_win/lose only if the corresponding HP hits 0 and stays terminal; (iii) no continued attack animation after HP=0. Report rule-pass rate for StatePlay vs fine-tuned ReactiveGWM. If the gap shrinks well below ~18% while LLM fidelity stays high, the headline claim weakens.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that joint MoT state–frame modeling improves mechanics fidelity by ~18.6% (ReactiveGWM 63.7% → StatePlay 82.3% Gemini; Tab. 1, abstract). That delta is measured almost entirely by Gemini-3.1-Pro and GPT-5.5 with a fixed prompt and reference images (§4.1, Appendix D). The same Gemini stack labeled training clips for result_win/lose and macro_success/fail and wrote NPC strategy text (§3.1, Appendix B). So the headline number can partly score agreement with the labeler–judge stack rather than true rule compliance in the SF3 engine.\n\nState alignment (normalized L1 < 0.06, score 0.947) is engine-grounded and strong, and qualitative Fig. 4/7 show clearer win/lose and super-art resets. But mechanics fidelity is defined as category match to target labels (win/lose/macro/normal), not as whether predicted HP/meter/timer trajectories and action-conditioned transitions match emulator memory on the generated rollout. Without that bridge, the 18.6% figure is the load-bearing but least secure support for “mechanics-consistent generation.” Ablations (Tab. 2) improve the same judge metric, so they do not fully break the circularity.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"StatePlay argues that game world models must jointly predict internal game states (timer, health, skill meters) and pixels to produce mechanics-consistent rollouts, rather than learning rules only implicitly from frames and actions. The authors build a synchronized SF3 state–frame–action dataset (10k clips, mechanics-balanced), and train a MoT-style dual-branch model (0.76B state expert + 5B visual expert on Wan2.2-TI2V) with joint attention (Eq. 3), flow-matching for video (Eq. 4), and Smooth-L1 regression for states (Eq. 5). On a 100-clip test set they report normalized state L1 below 0.06 (alignment 0.947) and mechanics fidelity of 82.3%/78.3% (Gemini/GPT) versus 63.7%/59.7% for the best fine-tuned visual-only baseline (ReactiveGWM), i.e. the advertised ~18.6% gain, with ablations favoring MoT over shared backbones and regression over flow matching for states (Tab. 2).","tokens_in":15981,"tokens_out":1657,"duration_ms":37277,"significance":"The paper correctly identifies a real gap: recent game world models optimize visual and action controllability but routinely violate state-dependent rules (super arts without meter, play after KO). Coupling an explicit, engine-readable state head with frame generation is a natural and transferable idea, and the MoT-plus-modality-specific-loss design is a concrete, reusable recipe. Strengths include a carefully balanced state-aware dataset, clear architecture (Fig. 3), grounded state-alignment numbers, informative ablations (Tab. 2), and qualitative cases where win/lose screens and meter resets track rules better than a strong in-domain baseline (Fig. 4/7). If the mechanics-fidelity claim holds under engine-grounded checks and broader games, this would push the field from pixel demos toward playable simulators. Credit is due for releasing a full training recipe and for showing that explicit state modeling need not destroy SSIM/LPIPS or action accuracy.","major_comments":[{"comment":"The headline 18.6% mechanics-fidelity gain (abstract; §4.3; Tab. 1: ReactiveGWM 63.7% → StatePlay 82.3% under Gemini-3.1-Pro) is measured almost entirely by LLM visual judges (Gemini-3.1-Pro and GPT-5.5) with the Appendix D prompt and reference images (§4.1). The same Gemini stack is used to label training clips for result_win/lose and macro_success/fail and to write NPC strategy text (§3.1, Appendix B). This creates a load-bearing circularity risk: the primary claim metric can reward agreement with the labeler–judge stack rather than true SF3 rule compliance. Please add an engine-grounded mechanics evaluation on generated rollouts—e.g., whether predicted or implied HP/meter/timer trajectories and termination/super-art events match emulator memory or deterministic rule checks conditioned on the commanded actions—and report agreement between LLM judges and those checks. Until then the 18.","section":"§4.1, §4.3, Tab. 1, Appendix D"},{"comment":"State alignment (normalized L1 < 0.06, score 0.947) is engine-supervised and strong, but the paper does not close the causal loop from accurate state prediction to mechanics-consistent pixels on the generated sequence. Mechanics fidelity is defined as category match to target labels (win/lose/macro/normal), not as consistency between the model’s predicted state trace, the generated HUD/events, and emulator transitions. Appendix C already notes HUD/state visual mismatches. A quantitative bridge—e.g., correlation of predicted HP/meter with rendered bars, or rule-violation rate under forced zero-HP / full-meter action sequences—would make the central “mechanics-consistent generation” claim much more secure.","section":"§4.1 State Alignment / Mechanics Fidelity; Appendix C"},{"comment":"All quantitative claims rest on a single game (SF3) and a 100-clip test set evenly covering mechanics categories (§4.1). Zero-shot baselines are largely out-of-domain and weak on visuals; the decisive comparison is fine-tuned ReactiveGWM on the same data (Tab. 1). LingBot-family fine-tunes are omitted for compute reasons. The directional MoT/regression ablations (Tab. 2) are informative but inherit the same judge and domain. The manuscript should either (i) add at least one additional title or a held-out character/rule split with engine checks, or (ii) clearly scope the contribution as a single-domain proof-of-concept and temper abstract/conclusion language about “complete and mechanically faithful game generation.”","section":"§4.1–4.3, §5, Tab. 1–2"}],"minor_comments":[{"comment":"Abstract and intro state “average normalized L1 distance below 0.06” while Tab. 1 reports only the aggregate alignment score 0.947. Please report per-variable normalized L1 (timer, P1/P2 HP, both meters) in the main table or appendix.","section":"Abstract; Tab. 1"},{"comment":"Eq. (1)–(2) use flow-style noise on states even though the state objective is regression (Eq. 5). Appendix E compares broadcast vs noise inputs, but the main text should briefly justify why noisy state inputs are retained under a non-flow state loss.","section":"§3.2; Appendix E"},{"comment":"ClipAttackNet (ResNet-18 + dilated TCN, conf. 0.7) and Move-Acc thresholds are custom; report validation accuracy of the attack classifier and the exact displacement thresholds so action-control numbers are reproducible.","section":"§4.1 Action Control"},{"comment":"Figure 1’s bottom “table” is hard to read in text form; ensure the camera-ready figure has legible axis labels. Fig. 4/7 would benefit from overlaid predicted vs GT state traces next to frames.","section":"Fig. 1, Fig. 4, Fig. 7"},{"comment":"Typos/spacing artifacts from PDF extraction appear throughout (e.g., “Recentgameworldmodels”, “StatePlayjointly”). Clean compound-word spacing in the camera-ready version.","section":"Throughout"},{"comment":"Related work cites WildWorld and From Pixels to States as closest prior datasets; a short explicit comparison of state schemas and whether those works train joint generators would sharpen novelty.","section":"§2.2"}],"recommendation":"major_revision","confidential_remarks":"The technical direction is sound and timely for a CV/ML venue that has been flooded with game world-model papers that ignore rules. I would not reject on novelty. The blocker is evaluation integrity of the headline metric (same LLM family for dataset labels and fidelity judging) plus single-game scope. If the authors add engine-grounded mechanics checks—even on SF3 only—and tone the abstract accordingly, this could become a clear accept/minor-revision paper. Fit is appropriate for cs.CV / world-models tracks."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"Punchline: this is a clean, useful fix for a real failure mode in game world models—pretty frames that break HP, super-art, and KO rules. The new piece is not “add states,” it is joint MoT-style dual-branch generation with regression on emulator states and cross-attention so predicted states can steer pixels, plus a balanced SF3 state–frame–action set.\n\nWhat they do well. State prediction is engine-grounded and strong (normalized L1 under 0.06, alignment ~0.95). The comparison that matters is fair: same data, fine-tuned ReactiveGWM as the strong visual-only baseline, and StatePlay still lifts mechanics scores while holding SSIM/LPIPS and action control. Ablations land where they should—MoT beats shared backbone on fidelity; Smooth-L1 beats flow-matching on low-dim rule variables. Figs. 4/7 make the qualitative case (meter reset, real win/lose screens) without hand-waving. Related work is honest about WildWorld / From Pixels to States stopping short of generator integration.\n\nSoft spots, in proportion. The load-bearing 18.6% (63.7→82.3 Gemini) is almost entirely LLM-as-judge, and Gemini also labeled train categories and NPC text. That is real circularity risk for the headline number; ablations move the same metric so they do not fully clear it. They underuse the obvious bridge: check whether generated rollouts’ HP/meter/timer traces still match emulator logic under the commanded actions. Single game, 5s clips, 100-clip test, no shipped code/data—expected for a first cut, but it caps how far the claim travels. Minor: occasional bar/UI drift even when latent states are right.\n\nWho it is for: people building interactive / game world models who care about playability, not pure video quality. Math is standard flow-matching + regression; citations look appropriate; no internal contradiction.\n\nI would send it to peer review. Ask for engine-backed fidelity on generated trajectories, multi-seed reporting, and ideally a second game or released artifacts. Worth engaging if you work in this lane; not a general CV must-read.","headline":"Solid subfield paper: explicit state heads actually help rule-consistent game rollouts on SF3, but the 18.6% headline leans too hard on an LLM judge that also labeled the data.","tokens_in":16604,"tokens_out":571,"would_cite":true,"duration_ms":20203,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Game world models need explicit internal states, not just pixels, to obey the rules.","keywords":["game world models","state-aware generation","mechanics consistency","mixture of transformers","action-conditioned video","Street Fighter 3","flow matching","interactive simulation"],"falsifier":"Run the same action sequences in the real SF3 emulator and in StatePlay; if win/lose screens, super-art gates, and HP/meter traces diverge often—or if a non-LLM judge (engine memory dump or human raters) erases the reported fidelity gap versus a fine-tuned pixel-only baseline—the central claim fails.","tokens_in":16423,"feed_emoji":"🎮","tokens_out":823,"duration_ms":19803,"temperature":0.7,"pith_summary":"Visually realistic game generators can still break the game: characters attack at zero health or fire super moves without a full meter. StatePlay argues that playable simulation requires jointly predicting the hidden rule variables—health, skill meters, timers—together with the frames. It builds a dual-branch mixture-of-transformers model on a new Street Fighter 3 dataset of synchronized frames, actions, and states, so predicted states can steer what appears on screen. The result is accurate state traces and substantially higher mechanics fidelity than strong baselines that only learn from pixels, without sacrificing visual quality or action control. The point for a sympathetic reader is simple: if you want generated games that are actually playable, you must model the state machine, not only the pixels.","feed_headline":"Game AIs break rules until they track health and meters","feed_subtitle":"Joint state–frame prediction lifts mechanics fidelity ~19% over pixel-only world models","key_machinery":"A mixture-of-transformers (MoT) dual branch: a lightweight state expert trained with Smooth L1 regression and a visual expert trained with flow matching, linked by bidirectional joint attention so states guide frames and frames inform states, with player actions injected into both.","core_discovery":"Explicitly coupling internal game-state prediction with visual generation produces rollouts that follow state-dependent rules far better than pixel-only game world models. On Street Fighter 3, StatePlay keeps average normalized state error below 0.06 and raises mechanics fidelity by about 18.6% over the best comparable baseline while holding visual quality and action controllability.","pith_inferences":["The same state-branch pattern should transfer to other genres where rules hide in counters (ammo, nitro, cooldowns), not only fighting games.","Closing the remaining HUD-vs-predicted-state mismatch may need tighter pixel supervision on bars and meters, not only latent state regression.","If LLM judges and LLM labelers share biases, future benchmarks will need engine-grounded oracles before claiming “playable” status."],"forward_implications":["Pixel-only game world models will keep producing rule-breaking rollouts even after in-domain fine-tuning.","Datasets that pair frames and actions with explicit states become necessary infrastructure for playable generators.","Modality-specific losses beat forcing compact rule variables through the same flow-matching objective used for video.","Separating state and visual experts while sharing joint attention is a practical recipe other game world models can adopt.","Mechanics-critical events (KO, super art, meter reset) should be oversampled when training interactive generators."],"fun_headline_variants":["State-aware models keep game rollouts inside the real rules","Joint state–frame prediction cuts mechanics errors 18.6%","Tracking health and meters stops game AIs from breaking rules","StatePlay holds normalized state error under 0.06 on SF3","Pixel-only world models violate mechanics; state coupling fixes it"],"cache_read_input_tokens":128,"weakest_assumption_plain":"That scoring “mechanics fidelity” with the same class of vision-language judges used to label the training set is a fair measure of true rule compliance rather than agreement with that labeling stack.","fun_headline_variants_meta":{"raw":{"variants":["State-aware models keep game rollouts inside the real rules","Joint state–frame prediction cuts mechanics errors 18.6%","Tracking health and meters stops game AIs from breaking rules","StatePlay holds normalized state error under 0.06 on SF3","Pixel-only world models violate mechanics; state coupling fixes it"]},"model":"grok-4.5","effort":"low","cost_usd":0.005024,"raw_usage":{"total_tokens":1395,"prompt_tokens":782,"num_sources_used":0,"completion_tokens":89,"cost_in_usd_ticks":50244000,"prompt_tokens_details":{"text_tokens":782,"audio_tokens":0,"image_tokens":0,"cached_tokens":128},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":524,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":782,"tokens_out":89,"duration_ms":9616,"temperature":1.0,"reasoning_tokens":524,"cache_read_input_tokens":128,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-30T22:03:31.092944+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Run the same action sequences in the real SF3 emulator and in StatePlay; if win/lose screens, super-art gates, and HP/meter traces diverge often—or if a non-LLM judge (engine memory dump or human raters) erases the reported fidelity gap versus a fine-tuned pixel-only baseline—the central claim fails.","supporting_citations":[],"review_version":1}