{"id":"75144d13-f0b3-4e20-93cd-847a1464a07b","arxiv_id":"2607.04409","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"Closed-loop agentic probing plus minimality/sufficiency masking recovers compact task-sufficient world-model latents that improve sample-efficient policy learning and cross-task generalization.","lead":"The paper introduces MIST-WM, a closed-loop method that pairs active probing exploration with structured masking so world models keep only task-sufficient latents. If it works as claimed, agents plan and transfer with far less irrelevant visual clutter, improving sample efficiency and skill/object generalization in control and robotics.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"Surrogate mask objectives do not guarantee formal MIST sufficiency; incomplete factor recovery and task-descriptor dependence leave the 'all control-relevant factors' claim under-supported.","rationale":"The reader's weakest assumption correctly isolates the load-bearing gap: formal MIST (DBN parents + conditional independence) versus the practical surrogates of Sec. 3.1. That gap directly undercuts the strongest claim's language of recovering MIST representations that capture all control-relevant factors. Empirical support (probing R², sample-efficiency curves, ablations approaching an oracle on some generalization panels) is real and multi-suite, and the paper is transparent that the objectives are surrogates; the contribution remains a coherent, useful closed-loop method. No stronger internal inconsistency or experimental fatal flaw appears that would push below CONDITIONAL. The proposed concrete test would settle whether the surrogate is merely a useful regularizer or is actually recovering a control-sufficient subspace in the formal sense. Scaffolding of multi-object environments (App. D.2) and reliance on gi are secondary amplifiers of the same concern, not independent load-bearing failures. Verdict therefore stays CONDITIONAL; no adjustment required.","tokens_in":27995,"tokens_out":724,"duration_ms":42025,"concrete_test":"On the RoboSuite stacking setup with known ground-truth factors, after full MIST-WM training, (i) train a policy that receives only the learned masked states mi⊙si and compare its return to an oracle policy given the true DBN parents of reward plus their one-step parents (or full simulator state); (ii) estimate residual mutual information I(rt; s_full \\ s_MIST | s_MIST, at) on held-out trajectories. If the MIST-only return gap exceeds ~15% of the oracle or residual MI is non-negligible, the claim that the surrogate captures all control-relevant factors fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The strongest claim asserts that the closed-loop recovers task-specific minimal sufficient (MIST) latents that capture all control-relevant factors, where MIST is defined formally as the DBN parents of reward and their one-step parents (Def. 1; sufficiency/minimality via conditional independence in App. B.1 Defs. 2–3). Section 3.1, however, optimizes only practical surrogates: reward likelihood under a soft mask (Eq. 2) and MI of masked states with a task descriptor gi minus an L1 penalty (Eq. 3), plus InfoNCE segment separability (Eq. 5). These do not enforce the conditional independencies of Defs. 2–3, do not explicitly target one-step dynamics parents of reward parents, and depend on gi (RoBERTa task embeddings when available, otherwise rewards). The paper itself labels this a 'practical surrogate' aiming at a 'MIST-aligned subspace rather than exact latent DBN variables.' Probing on the controlled RoboSuite stack (Fig. 4, Table A9) shows high R² on several factors but incomplete recovery (gripper openness 0.21, size ≈0). Consequently, the assertion that the synergy recovers all control-relevant factors rests on unproven alignment between surrogate and formal MIST, not on a guarantee or exhaustive check that residual reward-relevant information is zero.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper proposes MIST-WM, a closed-loop framework that couples agentic exploration with structured world-model learning to obtain task-specific, minimal, and sufficient latent states for control. On the agent side, a skill library (DIAYN/METRA) is refined by an InfoNCE segment-separability objective and an adaptive UED-style curriculum to collect informative trajectories. On the model side, Dreamer-v3 latents are gated by a soft mask optimized for reward likelihood (sufficiency) and mutual information with a task descriptor minus an L1 penalty (minimality), with an expandable latent dimension across a task sequence. The authors evaluate representation recovery via linear probes to simulator factors on RoboSuite, single-task sample efficiency on Meta-World and DMControl, and skill/compositional/unseen generalization on RoboSuite, Franka-Kitchen, and Meta-World, with multi-seed results and ablations over exploration, structure terms, backbone, and curriculum.","tokens_in":28463,"tokens_out":1541,"duration_ms":21315,"significance":"If the empirical claims hold, the work is a useful contribution to model-based RL: it targets task-sufficient rather than generic or fully factored latents, and shows that closed-loop data curation plus structure-aware masking can improve sample efficiency and transfer on standard continuous-control and manipulation suites. Strengths include multi-benchmark evaluation (DMControl, Meta-World, RoboSuite, Franka-Kitchen), five-seed reporting, direct R² probing against ground-truth factors, and ablations that isolate exploration, mask/MI/sufficiency terms, backbone, and curriculum. The formal MIST definition (DBN parents of reward and one-step parents) is a clear conceptual target even if the learning objectives are surrogates. The package is of interest to the world-model and unsupervised-skill communities, provided claims about recovering all control-relevant factors are calibrated to the evidence.","major_comments":[{"comment":"Abstract and Sec. 1 claim empirical recovery of task-sufficient latents that 'capture all control-relevant factors,' with MIST formally defined via DBN parents of reward and their one-step parents (Def. 1; App. B.1 Defs. 2–3). Sec. 3.1 instead optimizes practical surrogates—reward likelihood under a soft mask (Eq. 2) and MI(masked states, gi) − λ_M‖m‖1 (Eq. 3)—and explicitly calls this a 'practical surrogate' for a 'MIST-aligned subspace rather than exact latent DBN variables.' These objectives do not enforce the conditional independencies of Defs. 2–3, do not target one-step dynamics parents of reward parents, and depend on gi (RoBERTa embeddings when available, else rewards). The central claim should be restated as recovery of a useful task-aligned subspace, or the paper should add a direct residual-sufficiency check (e.g., reward prediction error with vs. without discarded coordinates","section":"Abstract; Sec. 3.1, Eqs. (2)–(3); Def. 1; App. B.1"},{"comment":"Fig. 4 and Table A9 show incomplete factor recovery on the controlled RoboSuite stack: gripper openness R²≈0.21 and size≈0, while cube mass/position/orientation and gripper pose are high. The text notes that static or fine-grained factors are hard to identify with contrastive pairs, yet still asserts that the subspace 'retains what matters for the task.' For the 'all control-relevant factors' claim, the paper should either (i) demonstrate that residual unprobed factors are irrelevant for return under the evaluated policies (e.g., policy performance when those coordinates are ablated or when ground-truth residual factors are added), or (ii) qualify the claim to the factors that are intervenable and reward-linked under the probing design. Incomplete recovery is not fatal to the empirical story, but it currently undercuts the strongest wording.","section":"Sec. 5, RQ1; Fig. 4; Table A9"},{"comment":"The adaptive curriculum and active-probing design rely on environment scaffolding that places objects with controlled factor differences (App. D.2: multi-cube mass/friction/orientation variants; multi-object Meta-World/Kitchen setups). Baselines receive the same phase-one budget, which is fair for comparison, but the method’s ability to expose latent factors may depend on this design. The paper should clarify how much of the representation recovery is attributable to the probing objective (Eq. 5) versus the curated multi-object scenes, and discuss applicability when such scaffolding is unavailable (e.g., single-object or real-world settings without factor-controlled variants).","section":"Sec. 3.2; App. D.2"}],"minor_comments":[{"comment":"Fig. 2(b) and Fig. 3 are dense; a short caption walkthrough of which modules are frozen vs. updated (cf. Table A2) would help readers follow the closed loop.","section":"Fig. 2–3; Table A2"},{"comment":"Notation for MIST indices mixes I^{(t)}_{i,1}, I^{(t)}_{i,2}, and U_i; a single consistent symbol for the selected subspace S^{min}_i in the main text would reduce load.","section":"Sec. 2, Def. 1"},{"comment":"Reacher-Hard is the main single-task underperformance vs. TD-MPC2 (Fig. 5, Table A11); a brief discussion of when structure-aware Dreamer objectives lose to model-free control objectives would strengthen the analysis.","section":"Sec. 5, RQ2; Fig. 5"},{"comment":"App. A limitations correctly flag simulation-only evaluation and scaling to diffusion world models; a sentence in the main conclusion pointing to these would set expectations for practitioners.","section":"Sec. 6; App. A"},{"comment":"Typos/consistency: 'task-specific, minimal, andsufficient' spacing in abstract; 'w/o su ciency' in Fig. 7; occasional Dreamer-V3 vs Dreamer-v3 capitalization.","section":"Abstract; Fig. 7"}],"recommendation":"major_revision","confidential_remarks":"The formal-vs-surrogate gap is the main load-bearing issue; the multi-benchmark empirical package is otherwise solid and above typical workshop quality. I would not reject on novelty grounds—the closed-loop framing and MIST target are a reasonable synthesis of factored world models, MISL, and UED—but the abstract’s 'all control-relevant factors' language should not survive without residual-sufficiency evidence or rewording. Fit for a top ML venue is plausible after revision."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The useful part of this paper is the closed loop: active probing skills (MISL + InfoNCE segment separability) under a UED-style curriculum, feeding Dreamer-v3 latents that are then masked by reward-likelihood sufficiency plus MI-to-task-descriptor minimality, with expandable dimension across tasks. That package is new as an end-to-end recipe even though every piece is known. Empirically it is stronger than the usual Dreamer / factored / DINO-WM baselines on Meta-World, RoboSuite, Kitchen, and most of DMControl, with 5-seed curves, R^{2} probing to simulator factors, skill/compositional/unseen transfer, and clean ablations. The representation-to-policy link in Fig. 4 is especially clear: different masks produce different, interpretable behaviors.\n\nThe stress-test is right on the formal gap and should not be waved away. Def. 1 and App. B.1 define MIST via DBN parents of reward and their one-step parents (conditional independence). Section 3.1 optimizes only surrogates (Eqs. 2–3, 5) and explicitly calls them a “practical surrogate” for a “MIST-aligned subspace.” Probing recovers cube/gripper pose and mass well but not gripper openness or size; nothing shows residual reward-relevant information is zero. Task descriptors (RoBERTa when available) and the heuristic Δd expansion are free parameters that matter. So the abstract’s “all control-relevant factors” is stronger than the evidence. That is a real soft spot, not a fatal one: the method still produces compact, useful latents that transfer.\n\nMath is standard variational + MINE + gated mask; citations are fair to TIA, Denoised MDP, I-Factor, METRA, UED. No code. Limitations (sim-only) are stated honestly.\n\nThis is for people building model-based robot agents who care about compositional transfer. It deserves a serious referee; the empirical package is already referee-ready, the formal claim needs tightening. I would engage with it and expect to cite the method once the surrogate language is cleaned up.","headline":"Solid closed-loop recipe for task-sufficient latents with real multi-suite gains; formal MIST claim is oversold relative to the surrogate objectives.","tokens_in":29086,"tokens_out":549,"would_cite":true,"duration_ms":7401,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Agents that actively probe environments and distill only control-relevant latents learn world models that transfer across skills and new tasks.","keywords":["world models","model-based RL","task-sufficient representations","minimal sufficient states","agentic exploration","skill discovery","adaptive curriculum","robotic manipulation"],"falsifier":"In a simulator where ground-truth factored states are known, train MIST-WM and check whether linear probes of the masked latents recover every ground-truth parent of the reward (and its one-step parents) with high R-squared while discarding non-intervenable factors; if key reward parents remain unaligned or non-intervenable factors are retained, the recovery claim fails.","tokens_in":28868,"feed_emoji":"🤖","tokens_out":601,"duration_ms":5754,"temperature":0.7,"pith_summary":"Standard world models keep too many visual or latent factors that do not matter for control, which hurts sample efficiency and transfer. This paper argues that agents can instead recover task-specific, minimal, and sufficient latent states—called MIST states—by closing a loop between purposeful exploration and structured model learning. The agent actively probes each new environment with skills chosen to expose latent factors that affect observations and rewards, under a curriculum that prioritizes hard tasks. From that data the world model learns a soft mask that keeps only the coordinates needed to predict rewards while discarding the rest. Policies that act only on these compact states reuse control-relevant factors and generalize to new skill combinations and previously unseen tasks on continuous-control and robotic-manipulation benchmarks.","feed_headline":"World models that keep only control-relevant latents transfer better","feed_subtitle":"Active probing plus structured masking recovers compact states that generalize across skills and new tasks","key_machinery":"MIST states: the coordinates of a Dreamer-style latent vector selected by a soft mask that maximizes reward likelihood (sufficiency) while maximizing mutual information with a task descriptor and penalizing L1 mass (minimality); the data that train this mask come from skills chosen by InfoNCE segment separability under an adaptive curriculum.","core_discovery":"A closed-loop synergy of agentic active probing (under an adaptive curriculum) and structured world-model learning recovers task-specific minimal sufficient (MIST) latent representations that capture all control-relevant factors; policies conditioned on those representations achieve improved sample efficiency and generalization across skills, object–skill compositions, and previously unseen tasks.","pith_inferences":[],"forward_implications":[],"fun_headline_variants":["Closed-loop probing recovers minimal control-relevant latents","Agentic curriculum plus structure yields task-sufficient states","World models distill only factors needed for control and transfer","Synergy of exploration and masking learns compact decision latents","Active probing finds minimal sufficient states that generalize skills"],"cache_read_input_tokens":16512,"weakest_assumption_plain":"That the practical mask-and-mutual-information objectives, plus contrastive skill selection, actually recover the formal minimal set of reward-relevant latent factors rather than a merely useful but incomplete subspace.","fun_headline_variants_meta":{"raw":{"variants":["Closed-loop probing recovers minimal control-relevant latents","Agentic curriculum plus structure yields task-sufficient states","World models distill only factors needed for control and transfer","Synergy of exploration and masking learns compact decision latents","Active probing finds minimal sufficient states that generalize skills"]},"model":"grok-4.5","effort":"low","cost_usd":0.00511,"raw_usage":{"total_tokens":1379,"prompt_tokens":743,"num_sources_used":0,"completion_tokens":79,"cost_in_usd_ticks":51100000,"prompt_tokens_details":{"text_tokens":743,"audio_tokens":0,"image_tokens":0,"cached_tokens":128},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":557,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":743,"tokens_out":79,"duration_ms":5839,"temperature":1.0,"reasoning_tokens":557,"cache_read_input_tokens":128,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-11T19:23:01.188028+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"In a simulator where ground-truth factored states are known, train MIST-WM and check whether linear probes of the masked latents recover every ground-truth parent of the reward (and its one-step parents) with high R-squared while discarding non-intervenable factors; if key reward parents remain unaligned or non-intervenable factors are retained, the recovery claim fails.","supporting_citations":[],"review_version":1}