{"id":"af489ac2-743b-4e89-ae04-73d0ba831b2b","arxiv_id":"2607.04464","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Operator-on-F—a probe-based comparison of a model's k-step latent pushforward to the environment's—tracks planning return on TD-MPC2 cheetah-run (Spearman -0.90) and separates architectures where reward checks fail.","lead":"Operator-on-F is a new diagnostic that checks a world model's internal predictions against what actually happens, read through a probe on a chosen observable surface. On a five-size sweep of TD-MPC2 on cheetah-run, it flags a 317M model that reward-prediction error barely distinguishes, and its ranking tracks planning return far better than Bellman residual or reward error.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Probe-faithfulness confound across the five TD-MPC2 sizes is untested: Table 2 omits probe R² promised in §2.2, and the §4.3 cross-arch probe controls don't cover the size-sweep correlation.","rationale":"The paper's strongest claim is that operator-on-F detects planning-relevant latent rollout error missed by unnormalized reward/Bellman checks. That claim is supported by a five-point correlation on released checkpoints. The most load-bearing assumption is not the math of the operator but the comparability of the probe readout across very different latent geometries. A probe is a function Z→F; if the 317M latent lives in a geometry that is harder to linearly read, the 2.62 error may say more about the probe than about the dynamics. The paper is unusually honest about this (Sec. 5) and provides strong cross-architecture controls, but those controls do not address the size-sweep, which is where the headline −0.90 lives. Section 2.2 says per-observable R² accompanies each number; Table 2 omits them, so the assurance is not actually delivered for the central table. The reader's weakest assumption identifies the same issue, and the requested check is feasible: restrict F to well-probed observables and/or use a probe-free latent error. If the 317M point survives both, the diagnostic is credible. Until then, the existing CONDITIONAL verdict is appropriate; the concern doesn't overturn the paper but prevents ACCEPT.","tokens_in":8524,"tokens_out":7593,"duration_ms":86712,"concrete_test":"On the same 1500 anchors and five TD-MPC2 sizes, recompute full-F operator error under the commonly-well-probed restriction (only observables with probe R²≥0.7 for every size, as in App. D) and with an MLP probe; report per-size R². Then compare the resulting Spearman(operator, return): if it remains ≤−0.80 and 317M remains the clear outlier (operator error at least 2× next-highest), probe asymmetry is not the driver. Also compute a probe-free baseline: normalized RMS of the raw latent error ||z_hat − z'|| (per-dimension standardized by the true spread) and check whether it produces the same ordering; if the probe-free metric ranks sizes differently from operator-on-F, the probe surface is doing the work.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The empirical core — size-sweep Spearman −0.90 and the 317M collapse — depends on operator-on-F being comparable across latent geometries. Different TD-MPC2 sizes have different latent dims/shapes (e.g., 19M is 768-dim), and linear readout fidelity can vary sharply with dimension/geometry; the paper itself says the probe is a confound (Sec. 5). Yet the probe-fairness controls (R²≥0.7 restricted F, MLP ablation, App. B/D) are run only for the LeWM vs TD-MPC2 comparison, not for the five-size sweep in Table 2. Section 2.2 promises 'report[ing] per-observable R² with each operator-on-F number,' but Table 2 gives no R² for any size, so the reader cannot check whether 317M's 2.62 is high because its dynamics are bad or because its latent is less linearly readable. Because 317M is the extreme anchor of the rank correlation, probe asymmetry alone could manufacture both the wide operator-error spread and the −0.90 correlation. The MLP-probe ablation in App. B only addresses probe family, not cross-size readout difficulty.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a new diagnostic, operator-on-F, for evaluating latent world models in model-based RL. The diagnostic compares the model's k-step latent pushforward to the environment's encoded next-state on a chosen observable subset F, normalizing per-functional errors by the across-anchor spread of the true pushforward. On five released TD-MPC2 mt80 checkpoints on cheetah-run, the paper reports that reward-prediction error varies only ~3× (0.028 to 0.091), while full-F operator error spans 0.28 to 2.62, with the 317M model showing the largest operator error and a planning-return collapse to 0.9. The rank correlation between operator error and return is -0.90 (anchor-bootstrap 95% CI [-0.90, -0.70]; leave-one-out ≥ -0.80). The diagnostic is also applied to a cross-architecture comparison between TD-MPC2 and a pure-SSL LeWM, where LeWM shows lower operator error with disjoint 95% CIs, and this ordering survives a 1-hidden-layer MLP-probe ablation. The paper positions the diagnostic as complementary to value-equivalence, not a replacement, and explicitly limits the claims to the studied environment and checkpoints.","tokens_in":8827,"tokens_out":5852,"duration_ms":65847,"significance":"If the diagnostic is sound, it addresses a real gap in world-model evaluation: reward-prediction and Bellman-residual checks, as conventionally reported, can be insensitive to planning-relevant latent rollout errors. The paper's strengths include measuring against external ground truth (encoded real next states), using each model's own planning-time predictor, reporting bootstrap and leave-one-out analyses, comparing against a persistence baseline, and providing a probe-family ablation for the cross-architecture result. The size-sweep result is intriguing but rests on n=5 in a single environment; the paper is appropriately cautious about this. The main weakness is that the central size-sweep correlation depends on probe faithfulness across different latent geometries, and the promised per-observable probe R² values are not reported for the size sweep, so the probe confound is not fully addressed for the headline result.","major_comments":[{"comment":"The paper promises per-observable probe R² alongside every operator-on-F number (§2.2, Appendix B), but Table 2 reports no R² for any of the five TD-MPC2 sizes. This is load-bearing: the size-sweep correlation (Spearman -0.90) and the 317M extreme (2.62 vs. 0.28–0.36) require comparable probe faithfulness across latent geometries. The restricted-F and MLP-probe controls in §4.3/Appendix B cover only the two-architecture comparison. Please report per-observable R² for each size and add a probe-family or restricted-F control for the five-size sweep; otherwise probe asymmetry across latent dimensions (e.g., 768-dim 19M vs. other sizes) could manufacture part of the operator-error spread.","section":"§2.2 / Appendix B / Table 2"},{"comment":"Appendix C states that the resolution gap between operator-on-F and the value-equivalence proxies is 'largely a normalization effect' and that the normalized value slice tracks return at Spearman -1.00, but Table 2 does not give per-size values for the normalized value/reward slices. These slices use the model's own reward and value heads and do not require a learned probe, so their per-size numbers would directly test whether the 317M collapse and the size ordering are artifacts of the PCA probe in full-F. Please include those rows in Table 2 or in an appendix table.","section":"§4.1 / Appendix C"}],"minor_comments":[{"comment":"The 'kernel-divergence operator' is mentioned without definition or reference. Define it or cite a source so the reader can interpret the Spearman +0.10 comparison.","section":"§4.2"},{"comment":"The 'persistence baseline' (1.33 for LeWM, 1.40 for TD-MPC2) is not defined. State how it is computed (e.g., predicting the current state/observation as the next state under the same actions).","section":"§D"},{"comment":"The phrase 'per-anchor PCA basis' is confusing, as the PCA is fit once on a held-out half of the anchor pool (Appendix A). Reword to clarify that the basis is dataset-wide but fit on a held-out split.","section":"§2.1"},{"comment":"Minor spacing issues: 'operator-on-Fcomplements', 'oncheetah-run' should be 'operator-on-F complements' and 'on cheetah-run'. Also, 'an order of magnitude above the 0.28–0.36 cluster' is not literally an order of magnitude (2.62/0.28 ≈ 9.4); rephrase to 'roughly an order of magnitude' or '7–9×'.","section":"Abstract / throughout"},{"comment":"The value slice F={r, V} uses the model's own reward and value heads as fixed functionals, but the general definition of the probe φ_F suggests a learned map. Clarify that for reward/value heads the functional is fixed, not learned, so the probe confound applies only to the PCA and observation-level F.","section":"§2.1"}],"recommendation":"major_revision","confidential_remarks":"The core idea is promising and the empirical reporting is generally transparent, but the omitted probe R² for the size sweep is a concrete gap that the stated limitation in §5 explicitly acknowledges. The authors should either supply the promised per-observable R² and a probe-family control for the five-size sweep, or substantially soften the cross-size claim. I do not see a circularity problem; the main risk is probe-fairness, which is addressable within scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea is worth your time: operator-on-F compares the model's own k-step latent pushforward to the environment's on a shared observable subset, and it does this without needing a reward head. That is a real gap in how latent world models are evaluated, and the paper gives you a concrete, probe-based estimator plus code and a clean statement of what it does and does not claim. The observation that reward-prediction error stays nearly flat across the TD-MPC2 sizes while a normalized pushforward error varies widely is genuinely interesting, and the 317M collapse is a striking data point. Credit where due: the paper ships code, reports bootstrap and leave-one-out analyses, runs an MLP-probe ablation, discloses the n=5, single-environment limitations, and explicitly calls the probe a confound. That is the right level of honesty for this kind of empirical proposal.\n\nThe soft spots are real and one of them is load-bearing. The headline rank correlation of -0.90 is a five-point statistic anchored on the 317M outlier. Leave-one-out says the rank holds at -0.8 if you drop any single size, but with n=5 that is still not much. More important, the stress-test is right: the paper promises per-observable R2 values alongside each operator-on-F number, and Table 2 does not include them. Without R2 for each of the five sizes, you cannot tell whether the 317M error of 2.62 reflects bad dynamics or a latent that is simply harder to read out with a linear probe. The authors note in Section 2.2 that unregularized probes degrade sharply on the 768-dim 19M latent, so readout difficulty is already a documented issue in this exact family. The restricted-F and MLP-probe controls only cover the cross-architecture comparison, not the size sweep, so the main empirical claim is under-supported as written. The bootstrap CIs also resample anchors, not model checkpoints, so they overstate precision in the way the reader's report says.\n\nThat said, the central argument is not bankrupt. The paper openly treats this as a preliminary diagnostic, and the normalization insight alone—that per-functional normalization recovers return ordering while unnormalized reward error does not—is valuable. The fix is straightforward: report per-size probe R2, add a direct latent-MSE baseline, run more seeds and environments, and give checkpoint-level error bars. This is a solid workshop-caliber or early-venue paper that becomes a useful tool if those gaps close.\n\nBottom line: yes, send it to peer review. The diagnostic is novel, the evaluation is honest, and the flaws are addressable with more data rather than fundamental. I would not cite it as established fact yet, but I would put it on the reading list for anyone working on world-model evaluation.","headline":"A genuinely new planning-time diagnostic that deserves a serious look, but the headline size-sweep correlation rests on a probe-faithfulness assumption the authors flag yet never actually verify on the size axis.","tokens_in":9301,"tokens_out":1733,"would_cite":true,"duration_ms":22389,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Reward checks miss the failure that kills planning","keywords":["latent world models","model-based reinforcement learning","value-equivalence","world-model evaluation","operator-on-F","planning-time diagnostic","TD-MPC2","self-supervised latent models"],"falsifier":"Re-run the size sweep using an alternative readout of F that is hand-designed or trained only on low-dimensional latents: if the 317M operator error falls into the 0.28–0.36 cluster, the -0.90 rank correlation collapses. Alternatively, sweep the MLP-probe width over the cross-architecture comparison: if the LeWM/TD-MPC2 ordering inverts at some width, the disjoint confidence intervals are probe-family artifacts.","tokens_in":8366,"feed_emoji":"🎯","tokens_out":3648,"duration_ms":39021,"temperature":0.7,"pith_summary":"The paper argues that standard world-model evaluation—reward-prediction error and Bellman residuals—can appear healthy while the latent rollout used for planning is badly wrong. It introduces a complementary diagnostic, operator-on-F, that compares a model's k-step latent prediction against the environment's encoded next state on a shared observable subset, normalizing each channel by its natural spread. On five released TD-MPC2 sizes on cheetah-run, reward-prediction error stays within a narrow band while operator error spans 0.28 to 2.62, and the largest model, with a planning return collapse to 0.9, has operator error an order of magnitude above the others. The rank correlation between operator error and return is -0.90, stable to leave-one-out removal. The paper contends the diagnostic complements value-equivalence rather than replacing it, and it also discriminates between architectures on a shared observation surface.","feed_headline":"Reward checks miss the failure that kills planning","feed_subtitle":"A pushforward diagnostic ranks world models by return where reward and Bellman checks stay flat.","key_machinery":"Operator-on-F estimates, per anchor (state, action sequence, true next state), the error |φ(ẑ_{t+k}) − φ(z′_{t+k})| normalized by the across-anchor standard deviation of the true pushforward φ(z′), then aggregates by RMS over functionals φ in a chosen observable set F and over anchors. F can be the value slice {r, V} using the model's own reward and value heads, or full-F which adds a held-out, singular-value-whitened PCA basis fit on the encoded next-state geometry; a probe—ridge regression with λ grid search, or a one-hidden-layer MLP as a control—maps the model's latent space onto F. The comparison runs at each model's native planning horizon, k = 5, with the model's own nonlinear latent","core_discovery":"On the released TD-MPC2 mt80 size sweep over cheetah-run, the conventional value-equivalence proxies—unnormalized reward-prediction error and Bellman residual—do not order the five models by return (Spearman -0.30 and -0.10), while the full-F operator-on-F error tracks return loss with Spearman -0.90. The 317M checkpoint is the decisive case: its operator error is 2.62, an order of magnitude above the 0.28–0.36 cluster, and its planning return collapses to 0.9, even though its reward-prediction error (0.091) stays within the same narrow [0.028, 0.091] range as the rest of the sweep. The diagnostic also returns disjoint 95% confidence intervals when comparing a pure-SSL latent world model (0.","pith_inferences":["If the probe confound is properly controlled, operator-on-F could be used as an online planning-time monitor: recalculated on recent transitions, it might flag latent-model drift or task shift while reward head error remains flat.","The same pushforward-on-observable-subset idea could be directed at task-relevant or safety-relevant features rather than reward and value, catching planning failures specific to those dimensions.","The cross-architecture gap, if it generalizes, suggests that architectures trained purely with self-supervised objectives may develop latent rollouts that are closer to the environment's encoded next states on this task—but a two-point, single-environment comparison is too narrow to infer a general trend.","A natural next test is to run the diagnostic on a reconstruction-anchored architecture as an intermediate point in the architecture spectrum, which would separate the probe effect from the latent-geometry effect more sharply."],"forward_implications":["Reward-prediction error alone can falsely reassure a planner: the 317M model has reward error within the sweep's normal range while its operator error is an order of magnitude worse and its return collapses.","Reporting both value-equivalence proxies and operator-on-F gives a more complete picture of world-model quality for model selection and for deciding when a latent model is safe to plan with.","The per-functional normalization by pushforward spread is load-bearing: unnormalized value error is numerically near-identical to the Bellman residual (Spearman +1.00), while the normalized value slice tracks return at Spearman -1.00.","Operator-on-F extends naturally to models without reward or value heads, such as purely self-supervised latent world models, by comparing on a shared observation-level F.","The diagnostic's ranking is stable to single-point removal (leave-one-out Spearman ≥ -0.80), so the -0.90 correlation is not carried by any one model size."],"fun_headline_variants":["Reward error misses the model failure that wrecks planning","Operator-on-F catches what reward checks miss in world models","Planning crash invisible to reward error? New diagnostic spots it","Operator-on-F error orders world models; reward error can't"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The probe reads a shared observable subset F comparably under different latent geometries; if probe capacity or regularization interacts with latent dimensionality, the operator-error gaps and the cross-architecture ordering could be probe artifacts rather than differences in dynamics quality.","fun_headline_variants_meta":{"raw":{"variants":["Reward error misses the model failure that wrecks planning","Operator-on-F catches what reward checks miss in world models","Planning crash invisible to reward error? New diagnostic spots it","Operator-on-F error orders world models; reward error can't"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000874,"raw_usage":{"total_tokens":3693,"prompt_tokens":894,"completion_tokens":2799,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":638,"completion_tokens_details":{"reasoning_tokens":2731}},"tokens_in":638,"tokens_out":2799,"duration_ms":21231,"temperature":1.0,"reasoning_tokens":2731,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T08:36:17.952947+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the size sweep using an alternative readout of F that is hand-designed or trained only on low-dimensional latents: if the 317M operator error falls into the 0.28–0.36 cluster, the -0.90 rank correlation collapses. Alternatively, sweep the MLP-probe width over the cross-architecture comparison: if the LeWM/TD-MPC2 ordering inverts at some width, the disjoint confidence intervals are probe-family artifacts.","supporting_citations":[],"review_version":2}