{"id":"9d2f2edc-b32d-4a7f-90b7-2d3754b0801d","arxiv_id":"2607.26712","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Enforcing rollout separation and frozen action readout keeps latent world-model futures action-sensitive, improving Minecraft planning and offline action recovery versus LeWM-style predictors.","lead":"ActSWM trains latent world models so predicted futures stay distinguishable under different actions, fixing a failure the authors call Context Collapse. That makes model-predictive control more reliable in long Minecraft tasks and helps recover controls from offline game videos.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"The load-bearing concern is the same as the reader's: recorded-vs-zero is both the hinge objective and the sole Q1 sensitivity metric, so large Δ may not prove planner-relevant separability.","rationale":"The paper cleanly isolates Context Collapse and couples multi-step prediction with two action-sensitivity constraints; closed-loop gains (stone 19/20 vs 10/20, pillar 17/20 vs 11/20) and Q3 CEM-over-random gaps are independent of the zero hinge and support an engineering contribution. The weakest link is still the shared recorded-vs-zero proxy in training and the main diagnostic, exactly as the reader flagged. That does not justify REJECT: ablations, frozen vs joint readout, and interactive success still favor the method. It does keep the verdict CONDITIONAL until sensitivity is shown under non-zero alternatives the planner actually searches, with tighter stats. No stronger internal inconsistency (e.g., in the Lipschitz separation lemma) is needed to carry the concern.","tokens_in":17130,"tokens_out":640,"duration_ms":13818,"concrete_test":"Re-run the Q1 step-drift protocol (H=32, Keval=32, same windows) on ActSWM and the strongest LeWM+Rollout baseline, replacing the zero rollout with (i) random sequences from the empirical action pool and (ii) CEM library chunks used in Mine Stone / Build Pillar. Report mean Δk and final-step gap. If ActSWM’s gap collapses toward baseline under these non-zero contrasts while remaining large only vs zero, the proxy overstates planner-relevant sensitivity.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that enforcing transition-separation (hinge on recorded vs all-zero rollouts plus frozen readout) yields planning-useful action-sensitive dynamics. The hinge (Eq. 5) and the Q1 diagnostic (Eq. 10, Fig. 3, Table 5) use the identical contrast: futures under dataset actions vs the constant all-zero sequence. That makes the headline 0.760 gap partly a direct optimization target rather than an independent test of “distinct control sequences induce separable latent consequences.” CEM planning (Appendix C) searches a discrete library of non-zero chunks (look-down+attack, forward, place+pitch, stack, etc.), not the zero sequence. If zero is an easy/atypical counterfactual—especially under frameskip-4 and sparse buttons—the large Δ and the closed-loop lifts could overstate controllability among the alternatives the planner actually ranks. The frozen-readout argument (Appendix E) is sound for local recoverability but does not close this gap: it constrains transitions, not multi-step separability under realistic non-zero alternatives. Joint-readout ablation shows the frozen design matters, and MineStudio gains are real, but the diagnostic remains circular with the training contrast.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper identifies Context Collapse in latent world models: autoregressive predictors can match encoded futures under recorded actions while producing nearly identical rollouts under alternative (here, all-zero) action sequences, undermining MPC. ActSWM addresses this via a transition-separation principle implemented as (i) multi-step JEPA prediction, (ii) a cosine hinge that separates recorded-action vs all-zero rollouts (Eqs. 5a–5b), and (iii) a frozen action readout on encoded and predicted transitions (Eqs. 8a–8c, Appendix E). On Minecraft VPT step-drift, ActSWM reports a step-31 action gap of 0.760 versus ~0–0.002 for LeWM-style baselines while retaining high GT similarity; in matched CEM planning on MineStudio it raises success (e.g., stone mining 19/20 vs 10/20); and in cross-game CEM action recovery it improves CEM-over-random gaps by up to ~16.5× with higher active-key accuracy in some domains.","tokens_in":17468,"tokens_out":1560,"duration_ms":35106,"significance":"If the result holds, the work cleanly separates prediction fidelity from planner-relevant controllability and gives a practical training recipe (hinge + frozen readout) that improves closed-loop open-world planning without abandoning latent JEPA efficiency. Strengths include a matched-backbone ablation suite (context length, multi-step rollout, joint vs frozen readout), an explicit Lipschitz separation argument for the frozen readout (Appendix E), external closed-loop MineStudio evaluation under a shared CEM planner, and a multi-game offline recovery protocol with a clear CEM-over-random gap. These make action sensitivity a falsifiable, operational property rather than only an auxiliary inverse-dynamics head. The contribution is incremental relative to WAM/inverse-dynamics and LeWM, but the diagnostic framing and planning gains are of clear interest to model-based control and game-agent communities.","major_comments":[{"comment":"The headline Context Collapse diagnostic and the training hinge use the same contrast. Eqs. (5a–5b) maximize separation between recorded-action and all-zero rollouts; Q1 then reports Δ_k = s_gt_k − s_0_k (Eq. 10, Fig. 3, Table 5) on that identical pair, including the claimed ~380× gap (0.760 vs ~0.002). This does not make the method circular with Q2/Q3, but it does make the strongest diagnostic number a near-direct optimization target rather than an independent test that “distinct control sequences induce separable latent consequences.” Appendix C’s CEM library (look-down+attack, forward/back, place+pitch, stack chunks, etc.) never ranks the constant zero sequence. Please add at least one Q1-style evaluation under non-zero alternatives drawn from the planner’s action-block distribution (or random non-zero sequences from the empirical pool), and reframe the recorded-vs-zero gap as a contr","section":"Method (Eqs. 5, 10); Q1; Fig. 3; Table 5; Appendix C"},{"comment":"Closed-loop gains are important external evidence, but the reporting mixes relative and absolute figures in a way that overstates robustness. The abstract claims “up to 45% on stone mining and 30% on pillar building,” while §Q2 text says “90.0% on stone mining and 54.5% on pillar building” (relative to LeWM’s base rates), and Table 8 gives 10/20→19/20 and 11/20→17/20 over only 20 trials with no confidence intervals or multiple random seeds for environment/CEM stochasticity. For the central claim that action-sensitive rollouts improve long-horizon interactive planning, please standardize absolute success rates, add binomial CIs or repeated seeds, and clarify whether the abstract’s percentages are absolute points or relative lifts.","section":"Abstract; Q2; Figure 4; Table 8"},{"comment":"Q3 supports steerability but is uneven and partially under-explained as evidence for the same mechanism. ActSWM’s Gap improves in all three games (Table 1), yet active accuracy falls on Apex (0.576→0.269) while rising sharply on GTA (0.050→0.761), and LeWM uses H=3 while ActSWM uses H=32 (Appendix D), confounding architecture/objective with context length. G-IDM’s zero active accuracy is a weak foil if it collapses to the inactive class. To tie Q3 to the transition-separation claim, please ablate H-matched LeWM vs ActSWM, report variance across the 15 windows, and discuss the Apex active-accuracy drop rather than only the best-case 16.5× Gap.","section":"Q3; Table 1; Appendix D"}],"minor_comments":[{"comment":"Figure 3 caption and panel labels are dense; the right-panel gap annotation “=0.760” is easy to miss. Consider a small table callout of GT / Zero / Gap next to the bar chart.","section":"Figure 3"},{"comment":"Notation switches between p_θ, q_ϕ0, and “IDM” in figure legends (Fig. 3 “+IDM”) without defining IDM in the main method text; align terminology with “action readout.”","section":"Figure 3; Method"},{"comment":"Related Work cites several 2026 arXiv preprints heavily; ensure camera-ready versions and consistent spelling of method names (LeWorldModel / LeWM).","section":"Related Work; References"},{"comment":"Typos / spacing artifacts from line breaks: “Wetherefore,” “refertothis,” “isenforcedasaconstraint,” “multimodalagentsystems,” etc., appear throughout the Introduction and Method; a full copy-edit pass is needed.","section":"Introduction; Method"},{"comment":"Appendix A notes idm.stop_grad=true is misleadingly named; a one-sentence clarification in the main text that gradients still flow into latents would help reimplementers.","section":"Appendix A; ActSWM Architecture"}],"recommendation":"major_revision","confidential_remarks":"The zero-vs-recorded circularity flagged by the stress test is real for the diagnostic narrative but does not sink the paper: MineStudio under a shared planner is genuine external evidence. I would not reject on that basis. The work is a solid systems/ML contribution for cs.RO or a learning-for-control venue; novelty vs concurrent inverse-dynamics/JEPA action papers (WAM, Delta-JEPA, DWM) should be watched at camera-ready, but the frozen-readout + rollout hinge combination and the planning results are enough for major-revision rather than reject. Moderate confidence: I did not re-run code."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"Punchline: this is a clean systems fix for a real planner failure mode. Latent predictors can look accurate under recorded actions while becoming almost blind to the action sequence; ActSWM forces multi-step rollouts to stay separable and locally action-recoverable, and the closed-loop numbers move.\n\nWhat is new is the framing plus the constraint design. Context Collapse is distinguished from plain error accumulation: longer context and multi-step training push GT cosine up (to ~0.97) while zero-action similarity stays glued to it (gaps ~0–0.002). The joint-readout ablation collapses prediction quality; freezing the readout keeps GT fidelity (~0.92) and opens a 0.760 gap. The Lipschitz separation argument in Appendix E is short and honest—it justifies freezing, not optimality. Matched-planner MineStudio gains are the load-bearing external evidence (stone 19/20 vs 10/20, pillar 17/20 vs 11/20). Cross-game CEM recovery gaps jump hard on GTA/Apex; that is useful beyond Minecraft.\n\nSoft spots, in proportion. The stress-test is right that recorded-vs-zero is both L_hinge and the Q1 headline metric, so the 0.760 figure is partly optimized rather than a pure independent test of “any distinct controls.” CEM actually searches non-zero action chunks, not zeros. That weakens the diagnostic more than the method: closed-loop success and CEM-over-random on real video are outside the hinge definition. Still, they should add at least one non-zero alternative contrast in Q1. Q3 active accuracy is mixed (Apex drops), N=20 planning trials with no error bars, and baselines are narrow (LeWM family, not a full Dreamer/TD-MPC sweep). Free parameters (m, λs, K, CEM libraries) are normal for this genre. Citations look current and fair; no code release.\n\nWho it is for: people building latent MPC / JEPA dynamics for games or offline control labeling. Not a theory reorganization. I would bring it to reading group, cite the failure mode and freeze-vs-joint result if I work in this lane, and send it to referees—ask for a non-zero sensitivity metric, error bars, and a sentence on the Q3 regressions. Worth engaging.","headline":"Solid engineering paper: names Context Collapse, freezes a readout + hinge to keep JEPA rollouts action-sensitive, and shows real MineStudio and multi-game recovery gains—with one diagnostic that overlaps the training contrast.","tokens_in":18116,"tokens_out":581,"would_cite":true,"duration_ms":10593,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Latent world models fail at planning when futures stop depending on actions; ActSWM keeps those futures separable.","keywords":["latent world models","action sensitivity","context collapse","model-predictive control","JEPA","Minecraft planning","action recovery","open-world games"],"falsifier":"Rerun the step-drift and CEM planning tests with non-zero alternative action sequences drawn from the same action-block library the planner searches; if ActSWM’s action gaps and task-success gains collapse while prediction similarity stays high, the central claim fails.","tokens_in":17987,"feed_emoji":"🎮","tokens_out":825,"duration_ms":16090,"temperature":0.7,"pith_summary":"Latent world models let agents plan by rolling out futures in a compact code space instead of pixels. This paper argues that matching future states is not enough: if different action sequences produce nearly the same predicted futures, a planner cannot tell good controls from bad ones. The authors name that failure Context Collapse—rollouts that still look like plausible futures while becoming almost insensitive to the actions they are conditioned on. ActSWM counters it with a transition-separation rule: keep alternative-action futures distinguishable over long horizons, and keep each local latent transition informative enough that the action can be recovered. They enforce this with a hinge that pushes recorded-action rollouts away from all-zero-action rollouts, plus a frozen action readout that forces transitions to stay action-discriminative. On Minecraft step-drift tests the action gap stays large; closed-loop MineStudio planning gains large success lifts on sustained tasks; and CEM can recover controls from offline videos across several games much better than baselines.","feed_headline":"World models that ignore actions cannot plan","feed_subtitle":"ActSWM keeps predicted futures separable under different controls, lifting long-horizon game planning.","key_machinery":"Transition-separation principle, implemented as multi-step prediction plus an action-contrastive hinge (recorded vs all-zero futures) and a parameter-frozen action readout on encoded and predicted transitions, so gradients reshape the latent dynamics rather than move a trainable decoder boundary.","core_discovery":"A planning-useful latent dynamics model must keep futures under different actions distinguishable and make the action of each local transition recoverable. Enforcing that as a constraint on latent rollouts—not only as an auxiliary head—mitigates Context Collapse, preserves large action-dependent rollout gaps, and improves long-horizon closed-loop planning and offline action recovery.","pith_inferences":["Any world-model planner that scores candidate sequences in latent space may need an explicit anti-collapse term whenever context is long enough to predict motion from history alone.","Frozen readouts as hard constraints may transfer to other inverse-dynamics regularizers where joint training lets the head absorb the loss without separating transitions.","If zero-action counterfactuals overstate separation, training against sampled planner-like alternatives could close the gap between the diagnostic and real CEM search."],"forward_implications":["Future-state similarity alone is an incomplete training and evaluation target for planning-oriented latent world models.","Closed-loop MPC in open-world games should improve when latent rollouts preserve action-dependent separation over long horizons.","The same constrained dynamics can support recovering player controls from offline gameplay video via search in latent space.","Diagnostics that report an action gap (recorded vs counterfactual actions) become a practical check for controllability, not only prediction quality."],"fun_headline_variants":["Context Collapse: when world models forget actions mid-rollout","ActSWM keeps action-different futures separable for planning","Latent rollouts must stay action-sensitive or planning fails","Enforce action recoverability as a latent rollout constraint","Action-sensitive world models fix long-horizon game planning"],"cache_read_input_tokens":128,"weakest_assumption_plain":"Treating recorded actions versus all-zero actions as the main contrast is assumed to stand in for the planner-relevant property that different real control sequences produce separable futures.","fun_headline_variants_meta":{"raw":{"variants":["Context Collapse: when world models forget actions mid-rollout","ActSWM keeps action-different futures separable for planning","Latent rollouts must stay action-sensitive or planning fails","Enforce action recoverability as a latent rollout constraint","Action-sensitive world models fix long-horizon game planning"]},"model":"grok-4.5","effort":"low","cost_usd":0.002258,"raw_usage":{"total_tokens":916,"prompt_tokens":755,"num_sources_used":0,"completion_tokens":64,"cost_in_usd_ticks":22584000,"prompt_tokens_details":{"text_tokens":755,"audio_tokens":0,"image_tokens":0,"cached_tokens":128},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":97,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":755,"tokens_out":64,"duration_ms":3506,"temperature":1.0,"reasoning_tokens":97,"cache_read_input_tokens":128,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-30T23:16:14.576586+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Rerun the step-drift and CEM planning tests with non-zero alternative action sequences drawn from the same action-block library the planner searches; if ActSWM’s action gaps and task-success gains collapse while prediction similarity stays high, the central claim fails.","supporting_citations":[],"review_version":1}