{"id":"52ee0c4e-465d-4dfe-ae3d-ea087933d349","arxiv_id":"2608.05720","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"PhyLatent shows that globally non-collapsed JEPA latents can still be unreliable, and that adding physical-state, invariance, and counterfactual-action supervision reduces collapse and improves MPC planning.","lead":"PhyLatent adds training-time constraints to JEPA world models so their latent spaces keep physically identical states close, distinct states apart, and alternative action futures separated. On Cube and TwoRooms it lowers measured collapse rates and raises MPC planning success, while Reacher and PushT stay comparable.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Identifiability diagnostic is built from the same hand-selected state vector that PSG is trained to predict, so the headline collapse-rate drop may partly reflect direct state regression rather than a general dynamics-relevant geometry.","rationale":"The paper is honest about using task-specific physical targets and lists this in the limitations section. My concern is not that privileged state information is used; it is that the headline identifiability diagnostic is not independent of that privileged information. The threshold-sensitivity results in Table 11 and the grouped ablations in Table 5 are useful, but they do not break the coupling because every variant still evaluates Cid on the supervised vector. A held-out-variable test would settle whether PhyLatent improves physical identifiability beyond the dimensions it is explicitly told to encode. Until then, the correct verdict remains the reader's CONDITIONAL; my read does not change that verdict, so I recommend no adjustment.","tokens_in":13437,"tokens_out":6263,"duration_ms":70081,"concrete_test":"Re-run the Cube experiment with PSG targets restricted to a proper subset of the Table 7 vector (e.g., only the six arm joint positions) and then compute Cid from Eq. 5 using the full simulator-state vector, including cube pose and gripper contact. If the full-vector identifiability collapse rate stays near the 6.71% baseline while the supervised-subset rate drops, the reported improvement is an artifact of optimizing the evaluation target. Also report the same held-out comparison for physical invariance and counterfactual dynamics, plus MPC success, to show the effect is not confined to the supervised axes.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing weakness is that the physical-identifiability evidence is coupled to the training objective by construction. Eq. 5 defines the collapse criterion Cid using dphys = ||s̄_i − s̄_j|| on the standardized simulator-state vector; Eq. 13 trains Hs(Z) and Hs(Ẑ) to regress exactly that same s̄. A large reduction in the 6.71%→0.95% identifiability collapse is therefore close to a check that the encoder can be tuned to encode the supervised state vector, not independent evidence that the latent space preserves physical state more generally. The Cid diagnostic cannot see any variable omitted from the target list in Table 7. This is not hypothetical: for TwoRooms the PSG target contains only agent position, with target and door states explicitly excluded, so a representation that tracks the agent but ignores goal-relevant variables can pass this diagnostic. The MPC gains are real downstream evidence and the baseline failure rates still support the negative claim that global non-collapse is insufficient, but the central positive attribution — that explicit dynamics-relevant constraints, rather than direct state supervision, fix identifiability collapse — is weaker than the headline implies.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies LeWorldModel, a JEPA world model regularized by SIGReg, and argues that a globally non-collapsed latent distribution is not sufficient for a usable world-model state space. It identifies three collapse modes—physical invariance collapse, physical identifiability collapse, and counterfactual dynamics collapse—and proposes PhyLatent, a set of five auxiliary training objectives attached to the shared JEPA graph. On OGBench-Cube, PhyLatent reduces the three failure rates from 15.60%, 6.71%, and 8.41% to 7.53%, 0.95%, and 4.62%, and improves MPC success from 70.0% to 78.1%, with further improvement on TwoRooms and roughly neutral results on Reacher and PushT. The central negative claim—that global non-collapse alone does not guarantee dynamics-relevant structure—is supported by the baseline failure rates; the paper's positive attribution of the improvements to the proposed constraints is weakened by the coupling between the diagnostics and the training objectives.","tokens_in":13747,"tokens_out":9128,"duration_ms":105566,"significance":"If the causal interpretation holds, the paper makes a useful contribution to the JEPA-world-model literature by converting an abstract concern about latent geometry into three concrete, computable collapse diagnostics and by showing that training-time structure can improve MPC in the same architecture and planner. The paper has several genuine strengths: the baseline and method share the same architecture, planner, and evaluation protocol; the experiments use public standardized datasets; Appendix C.2 provides threshold-sensitivity evidence for the identifiability diagnostic; and the authors report a neutral PushT result and an explicit limitations section rather than overselling all four tasks. The main gap is that the headline diagnostic improvements largely measure quantities that the auxiliary losses directly optimize, so the evidence for a general 'dynamics-relevant geometry' benefit is weaker than the abstract suggests.","major_comments":[{"comment":"The physical invariance diagnostic and the SVIC objective are coupled. Eq. (4) defines dnuis = ||zt − z_aug|| for appearance perturbations, and Eq. (12) minimizes dN(Z_aug, sg(Z)), the normalized version of the same displacement under the same brightness/channel augmentation procedure. The text in §3.1 says the diagnostic uses brightness, color, texture, and image noise, but Appendix B.3 states that SVIC training uses only brightness and per-channel color offsets. Please clarify which perturbations the reported 15.60%→7.53% reduction uses; if texture and noise are included, report those failure rates separately, and if only brightness/color are used, the improvement is partly a check that the encoder can ignore the exact augmentation family minimized in training. A held-out invariance test on augmentations not used by SVIC would make the claim general.","section":"§3.1 (Eq. 4), Eq. (12), Appendix B.3"},{"comment":"The physical identifiability criterion Cid uses dphys = ||s̄i − s̄j|| on the standardized simulator-state vector, which is exactly the vector that PSG in Eq. (13) trains Hs to regress. The diagnostic therefore cannot detect omission or mis-weighting of a variable outside the hand-selected target list; on TwoRooms, the PSG target contains only agent position while target and door states are explicitly excluded, so a representation that ignores goal-relevant variables can pass Cid. At minimum, Cid should be re-evaluated on a held-out subset of physical variables not supervised by PSG, and PhyLatent should be compared with a direct state-regression-only baseline. The ablation in Table 5 complicates the picture: removing PSG+FRA leaves Id. Fail. at 1.03%, nearly unchanged from the full model, so the Cid reduction is not exclusively due to PSG; given the quantile-based thresholds in Eq. (5), the authors should analyze what drives the improvement in all three ablation rows.","section":"§3.2 (Eq. 5), Eq. (13), Table 7"},{"comment":"The counterfactual dynamics criterion Ccf directly uses the predicted branch distance dpred, and CASC trains a margin on the same normalized predicted-branch distance δz, with a mask that selects action differences above the batch median. The reduction from 8.41% to 4.62% is therefore partly a check that the optimization objective moved the measured quantity, not an independent test of counterfactual dynamics. The paper would be substantially stronger if Ccf were also reported on action pairs excluded by the CASC mask (e.g., below-median action differences) and as a function of action difference, so the diagnostic is not aligned with the training selection rule.","section":"§3.3 (Eq. 7), §4.3 (Eqs. 21–24)"},{"comment":"The three collapse rates are the headline quantitative results, but they are reported only as point estimates averaged over three random seeds, with no standard deviations or per-seed values. Since the same runs report error bars for MPC success in Table 4, the absence of variability information for the diagnostics makes it difficult to assess whether the reductions are statistically meaningful. Please provide per-seed diagnostics or standard deviations.","section":"Tables 1, 10 and Fig. 4"}],"minor_comments":[{"comment":"The text discusses the planning drop from removing PSG+FRA but does not discuss why Inv. Fail. (7.17) and Id. Fail. (1.03) are essentially unchanged or better in that row; given the central role of these diagnostics, a sentence explaining the ablation behavior would be helpful.","section":"§5.4, Table 5"},{"comment":"The exact Cube diagnostic numbers are duplicated in Fig. 4 and Table 10; it would be clearer to state in §5.1 that the reported diagnostics use the same augmentation ranges as training, or to specify the diagnostic augmentation distribution if it differs.","section":"§5.1 and Appendix C.1"},{"comment":"The threshold sensitivity analysis is useful, but the thresholds in Eq. (5) are quantiles of the model-dependent pair-distance distributions; reporting results with fixed absolute thresholds in addition to quantile thresholds would strengthen the claim that the improvement is not an artifact of the threshold choice.","section":"Appendix C.2"},{"comment":"There are several minor formatting/reference issues, including repeated spacing in author names such as 'Y ann LeCun' and the phrase 'V ertical axis' in Fig. 2; these should be cleaned up, but they do not affect the technical content.","section":"General"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things up front. First, this paper is worth a serious referee: it identifies a real blind spot in JEPA world models and shows, with the same architecture and planner, that fixing it improves MPC success on Cube and TwoRooms. Second, the most impressive-looking number, the identifiability collapse drop from 6.71% to 0.95%, is weaker evidence than it appears, because the diagnostic is built from the same hand-selected simulator state vector that PSG is trained to regress.\n\nWhat is actually new: the three failure-mode diagnostics (invariance, identifiability, counterfactual dynamics) are a useful contribution on their own. The observation that global non-collapse does not imply local physical structure is well demonstrated, and the negative result is convincing. The five-component objective is a sensible engineering package, and the ablations show each group contributes something. Credit where due: the same backbone, same planner, same seeds, and the MPC improvements are real downstream evidence. The PushT null result is reported honestly, which I respect.\n\nThe soft spots, in proportion. The identifiability diagnostic is partially circular: Eq. 5 uses the standardized simulator-state vector s̄, and Eq. 13 trains Hs to regress exactly that s̄. So the 0.95% rate partly measures how well the encoder can be tuned to encode a supervised target, not a general property of the latent geometry. The stress-test note is right about TwoRooms: PSG only targets agent position, so a representation that ignores the door and target can pass the diagnostic. The invariance and counterfactual diagnostics are less circular, but they mirror the training objectives too, and no error bars are reported for any diagnostic rate. The threshold robustness table helps, but it only varies the identifiability thresholds, not the target list.\n\nThose caveats do not sink the paper. The MPC gains on Cube (70.0 to 78.1) and TwoRooms (81.0 to 98.0) are downstream and not circular. The central negative claim, that SIGReg-style global regularization is insufficient, is supported regardless of the diagnostics. What is weaker is the positive attribution: that the dynamics-relevant constraints, rather than direct state supervision, fix identifiability collapse. I would want to see an ablation that swaps PSG targets for a random or incomplete state vector to test that attribution. No code is released, which matters for a benchmark-style paper, but that is fixable.\n\nWho gets value: anyone working on JEPA world models, latent planning, or representation collapse in control. It deserves peer review; my verdict would be conditional acceptance with a request for the state-target ablation, error bars on diagnostics, and code release. I would cite the negative result even while being skeptical of the identifiability headline.","headline":"A solid, honest JEPA-world-model paper whose headline collapse-rate drops are partly baked into the training objective, but whose downstream MPC gains and negative result about global non-collapse hold up.","tokens_in":14253,"tokens_out":682,"would_cite":true,"duration_ms":9407,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper argues that preventing global latent collapse in JEPA world models does not ensure a reliable state space, and that explicit dynamics-relevant constraints are needed.","keywords":["JEPA world models","latent collapse","dynamics-relevant representation","physical state grounding","counterfactual dynamics","model predictive control","representation learning","visual control"],"falsifier":"Train PhyLatent on Cube with the PSG physical-target sequence shuffled in time (each latent regressed to the state of a random other timestep) while keeping every other loss unchanged; if Cube success and the three failure rates stay close to the reported values, the gains do not come from state grounding, and if they revert to baseline, PSG is the load-bearing component.","tokens_in":13247,"feed_emoji":"🤖","tokens_out":8453,"duration_ms":74407,"temperature":0.7,"pith_summary":"JEPA world models learn to predict latent futures without reconstructing pixels, but the paper claims that keeping the latent distribution from collapsing globally is not enough to make it a reliable state space for planning. It identifies three concrete failures in the base LeWorldModel: appearance-only changes displace latents more than real state transitions, physically distinct configurations fold into the same latent neighborhood, and different actions produce predicted future branches that are too close together. PhyLatent adds three training pathways that constrain physical invariance, physical identifiability, and counterfactual dynamics, and on Cube it reports cutting all three failure rates substantially while raising model predictive control (MPC) success from 70.0% to 78.1%. The authors conclude that dynamics-relevant latent structure, not merely global non-collapse, is the deciding factor for a useful JEPA world model.","feed_headline":"JEPA planning gains 8 points when latents track physics","feed_subtitle":"Grounded latents cut three failure rates and lift Cube planning success from 70% to 78%.","key_machinery":"The load-bearing mechanism is a set of five training-time losses attached to the shared JEPA encoder–predictor graph, organized into three pathways. Physical State Grounding (PSG) regresses both the encoded and the predicted latent sequences onto standardized simulator-derived state vectors through a shared state head. The Static Visual Invariance Constraint (SVIC) forces an appearance-augmented sequence to have the same latent representation as the original. Future Representation Alignment (FRA) aligns the predicted future with the observed future through a shared projector and an action-query attention module. The Counterfactual Action Separation Constraint (CASC) imposes a margin between predicted future branches proportional to the difference in the actions, and Latent Denoising (LD) trains a denoiser on the noisy observed future to regularize local geometry. All auxiliary modules are discarded after training, so the inference-time model is identical to the baseline except for the trained weights.","core_discovery":"On the paper's own terms, the central discovery is that a JEPA latent space can be globally well spread—LeWorldModel's SIGReg keeps the overall distribution non-degenerate—and still be locally wrong where it matters for control: the same physical state maps apart under appearance noise, distinct physical states map close together, and alternative action futures are compressed. The paper diagnoses all three failures with quantitative criteria and then shows that PhyLatent reduces each failure rate on Cube, from 15.60% to 7.53%, from 6.71% to 0.95%, and from 8.41% to 4.62%, while improving planning success from 70.0% to 78.1% with the same architecture and MPC planner. Because the auxiliary heads are removed at inference, the gains come from the learned geometry of the shared encoder and predictor rather than from extra capacity at test time. The paper's conclusion is that enforcing dynamics-relevant relationships during training is necessary, and that global non-collapse alone is insufficient for a trustworthy JEPA world-model state space.","pith_inferences":["A natural next step the authors do not pursue is replacing simulator-state targets with pseudo-supervision from inverse dynamics or temporal contrastive learning, which would extend PhyLatent beyond tasks where ground-truth physical state vectors are available.","The binary collapse thresholds (such as $r_{cf} < 0.5$) could be converted into a continuous scalar 'dynamics-relevance score' by averaging the underlying ratios, giving a smoother evaluation signal and possibly revealing partial collapses.","The same three diagnostics could be re-run on other action-conditioned JEPA predictors, such as video-based ones, to test whether these failure modes are caused by the architecture or by the lack of explicit dynamics constraints.","The PushT gap suggests pairing PhyLatent with a contact-aware planning cost or a finer action sampling scheme may translate its latent improvements into control gains in contact-rich tasks."],"forward_implications":["If the central claim holds, JEPA world models that rely on global regularization alone are expected to make systematic planning errors wherever these three local collapses occur.","The three diagnostics (invariance failure, identifiability collision, counterfactual under-separation) offer a concrete, measurable way to audit a world-model latent space beyond collapse prevention.","Because the same architecture and planner improve from 70.0% to 78.1% on Cube and from 81.0% to 98.0% on TwoRooms, the benefit transfers across manipulation and navigation tasks.","The PushT result—lower failure rates but flat planning success—shows the method's gains are not automatically a planning gain in contact-rich tasks, which the paper attributes to local contact transitions and the planning objective."],"supporting_citations":[{"why":"Defines the Joint-Embedding Predictive Architecture that the paper diagnoses and extends.","marker":"LeCun, 2022"},{"why":"Provides the LeWorldModel baseline whose latent geometry is tested and later augmented by PhyLatent.","marker":"Maes et al., 2026a"},{"why":"Introduces SIGReg, the global Gaussian regularization that PhyLatent keeps at weight 0.09.","marker":"Balestriero & LeCun, 2025"},{"why":"Establishes action-conditioned JEPA prediction as a path to embodied planning, the setting PhyLatent targets.","marker":"Assran et al., 2025"},{"why":"Supplies the standardized datasets, environments, and evaluation interfaces for all four tasks.","marker":"Maes et al., 2026b"},{"why":"Provides OGBench-Cube, the benchmark where the three collapse diagnostics and main MPC results are measured.","marker":"Park et al., 2025"},{"why":"Supplies the PushT task used in the cross-task planning evaluation.","marker":"Chi et al., 2023"},{"why":"Supplies the Reacher task from dm_control used in the cross-task planning evaluation.","marker":"Tunyasuvunakool et al., 2020"}],"fun_headline_variants":["Physics-grounded latents lift JEPA planning from 70% to 78%","JEPA latents need physics, not just spread","PhyLatent: cut JEPA failure rates, boost control to 78%","Global non-collapse isn't enough: JEPA needs PhyLatent"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method's gains rest on the belief that the hand-picked simulator-state vectors used by Physical State Grounding are a complete and correctly scaled description of every variable that actually influences future behavior.","fun_headline_variants_meta":{"raw":{"variants":["Physics-grounded latents lift JEPA planning from 70% to 78%","JEPA latents need physics, not just spread","PhyLatent: cut JEPA failure rates, boost control to 78%","Global non-collapse isn't enough: JEPA needs PhyLatent"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000747,"raw_usage":{"total_tokens":3355,"prompt_tokens":995,"completion_tokens":2360,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":611,"completion_tokens_details":{"reasoning_tokens":2278}},"tokens_in":611,"tokens_out":2360,"duration_ms":17931,"temperature":1.0,"reasoning_tokens":2278,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T00:33:39.094755+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train PhyLatent on Cube with the PSG physical-target sequence shuffled in time (each latent regressed to the state of a random other timestep) while keeping every other loss unchanged; if Cube success and the three failure rates stay close to the reported values, the gains do not come from state grounding, and if they revert to baseline, PSG is the load-bearing component.","supporting_citations":[],"review_version":1}