{"id":"448df4a2-ca3d-4d8a-9511-f9bf666654f8","arxiv_id":"2608.07420","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A direct endpoint-prediction world model trained on long-horizon objectives substantially beats recursively-rolled-out baselines, and the objective, not the backbone, drives the improvement.","lead":"This paper shows that training a world model to directly predict the final observation after a long action sequence gives much better long-horizon accuracy than training it one step at a time and rolling out. The authors find the training objective, not the network architecture, is the main driver of the gains.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Central claim rests on a single unverified ablation: Table 2's ADM K=3 vs K=100 comparison may conflate training horizon with K-specific tuning, and the pixel benchmark lacks the short-horizon baseline needed to test the objective-vs-backbone claim.","rationale":"The paper's stated central claim is that the training objective, rather than the particular backbone, is the main driver of long-horizon prediction accuracy. The strongest direct evidence is Table 2, which retrains ADM with a long-horizon endpoint objective and compares against the published short-horizon ADM. This comparison is the load-bearing piece: without it, the DPWM architecture alone cannot separate objective from architecture, because DPWM's non-recursive single-forward-pass inference differs from ADM's autoregressive rollout in both training and deployment. The reader's weakest-assumption analysis correctly identifies the same risk: ADM K=3 and ADM K=100 are claimed to differ only in training horizon, but this is not verified with training curves, hyperparameter records, or multiple seeds. I also note that the pixel benchmark, which the abstract cites as supporting the claim, contains no short-horizon baseline in the reported quantitative results, so it cannot test the objective effect. None of this shows the claim is false; it shows that the current evidence is too narrow and insufficiently controlled for the strong abstract-level statement. A multi-task, multi-seed, fixed-budget replication of the Table 2 ablation would settle the question. The recommended verdict remains CONDITIONAL: the paper should be accepted only after the central decomposition is demonstrated with the missing controls, not rejected outright, because the proposed paradigm is coherent and the existing results are directionally consistent.","tokens_in":13251,"tokens_out":6942,"duration_ms":68216,"concrete_test":"Run the controlled ablation of Table 2 on at least two additional state-based tasks (e.g., cheetah_run and walker_run) and on Pong, with 5 random seeds, reporting mean and standard deviation, and include the ADM K=3 baseline in the pixel results. Fix a single hyperparameter budget (learning rate, gradient clipping, epochs) for both ADM K=3 and ADM K=100, and release training curves to confirm stable convergence; also verify that the encoder and rollout-generation code paths are byte-identical between the two settings. If ADM K=100 consistently beats ADM K=3 by similar margins across tasks with no task-specific tuning, the objective claim is supported; otherwise it is confounded.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"Table 2 is the only experiment that directly supports the central claim that the training objective, not the backbone, drives long-horizon accuracy. It compares ADM K=3 and ADM K=100 on a single task (humanoid_walk) and does not report seeds, error bars, a hyperparameter search, or optimization curves for the K=100 retraining. The paper states that ADM K=3 and ADM K=100 'share the same encoder and inference procedure, differing only in the training horizon' (Sec. 3.2), but this is asserted rather than verified: if K=100 training required a different learning rate, gradient clipping, or early stopping, or if the rollout generation for ADM behaves differently at long horizons, the observed gains could be attributable to training details rather than to the endpoint objective. Moreover, the DPWM-versus-ADM comparison is not architecture-controlled: DPWM uses a single non-recursive forward pass while ADM is evaluated by autoregressive rollout, so that contrast conflates objective, inference paradigm, and backbone. The paper's own Appendix D reports pixel Pong endpoint MSE only for DPWM K=100 vs ADM K=100, both long-horizon; there is no one-step or short-horizon baseline on pixels, so the objective-vs-backbone claim is not tested on the second benchmark. Thus the abstract-level claim generalizes from a single unverified ablation.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript argues that world models are conventionally trained with local one-step or few-step objectives and then evaluated by recursive rollout, which mismatches the long-horizon use case. It proposes the Direct Prediction World Model (DPWM), a non-recursive architecture that encodes the action sequence into a single embedding and predicts the endpoint observation in one forward pass. The paper derives a Lipschitz-based error bound motivating direct prediction (Appendix A), describes a practical architecture with standard components, and reports endpoint MSE on four DeepMind Control Suite tasks and on Pong. A controlled experiment retrains the ADM baseline under the same endpoint objective (Table 2) and is used to support the central claim that the training objective, rather than the backbone, drives long-horizon prediction accuracy.","tokens_in":13576,"tokens_out":6211,"duration_ms":58482,"significance":"If the central claim survives closer empirical scrutiny, the paper makes a useful contribution: it identifies a train/evaluation mismatch in world modeling, proposes a clean paradigm, and gives a practical architectural pattern that is not tied to a specific backbone. The Lipschitz derivation in Appendix A is clean and is correctly framed as a motivation rather than a guarantee. The experimental design of retraining ADM under Kmax=100 is the right way to isolate objective from architecture, and the qualitative Pong rollouts are suggestive. The main weakness is that the empirical support for the abstract-level claim is currently narrow: the controlled ablation is single-task, main tables lack a pure one-step autoregressive baseline, and no uncertainty quantification is reported anywhere.","major_comments":[{"comment":"The central claim of the paper is supported by a single-task ablation without uncertainty quantification. Table 2 compares DPWM K=100, ADM K=100, and ADM K=3 on humanoid_walk only; at K=1 ADM K=3 is actually better than ADM K=100 (0.1385 vs. 0.3448), and no confidence intervals or repeated-seed results are reported, so the K>=16 advantage cannot be judged statistically significant. The statement that ADM K=3 and ADM K=100 'share the same encoder and inference procedure, differing only in the training horizon' is asserted but not verified: no learning-rate schedule, gradient-clipping configuration, early-stopping criterion, or training curves are reported for the K=100 retraining, and long-horizon unrolled training may require different hyperparameters. Please add multiple seeds with error bars, at least two additional tasks, and full tuning details for the retrained baseline.","section":"§3.2, Table 2"},{"comment":"The main results table lacks a pure one-step autoregressive world-model baseline. Section 3.1 says 'Autoregressive baselines are trained with local one-step prediction losses,' but Table 1 contains only DPWM K=100, ADM K=3, and MoSim; ADM K=3 is a short-horizon any-step model, not a one-step recurrent model, and MoSim is a physics-informed ODE model. Consequently Table 1 does not test the objective-versus-backbone claim at all; only Table 2 does, and only on one task. Please add a recurrent one-step baseline (e.g., ADM K=1 or a standard RSSM) evaluated by autoregressive rollout, or explicitly reframe the headline comparison as an accuracy comparison rather than an objective comparison.","section":"§3.1, Table 1"},{"comment":"The pixel-based Pong experiment does not test the training-objective claim. Figure 4 is qualitative, and Table 4 compares DPWM K=100 with ADM K=100, both of which are trained under the same long-horizon endpoint objective; this comparison controls for backbone but cannot reveal whether endpoint supervision is better than short-horizon training. The paper therefore has no short-horizon baseline on the second benchmark, so the abstract-level generalization is untested on pixels. Please add a short-horizon (K=1 or K=3) recurrent baseline on Pong with rollout evaluation, and report seeds and error bars for Table 4.","section":"§3.3 and Appendix D"},{"comment":"The Kmax selection results are presented as single curves without uncertainty, yet they are used to draw conclusions such as 'Kmax=50 performs well only within its training horizon' and 'Kmax=400 yields consistently lower error' under random-policy evaluation. These claims would be substantially strengthened by reporting at least three seeds with confidence bands, and by specifying the horizon sampling distribution (uniform vs. log-uniform) and the random-segment intermix ratio, which are design choices that directly affect the curves.","section":"§3.2, Figure 3"}],"minor_comments":[{"comment":"The sentence introducing 'one-step prediction models' is dangling, because no such model appears in Table 1; please either add the baseline or revise the text so the reader can identify which rows correspond to one-step training.","section":"§3.1"},{"comment":"The gradient expansion in Eq. (5) is informal because the intermediate states s_j are themselves functions of theta; it would be helpful to state explicitly that this is a schematic unrolled-gradient expression rather than a complete derivation.","section":"§2.2, Eq. (5)"},{"comment":"The sup-norm error bound in Eq. (9) is a useful motivation, but the connection to the experiments would be stronger if the analysis were also phrased in expectation over the data distribution, since the reported results are MSE averages over trajectories.","section":"Appendix A"},{"comment":"The horizon distribution (uniform vs. log-uniform) and the random-segment intermix ratio are mentioned as design choices but are never given numerical values; specifying them would improve reproducibility.","section":"§3.2, Table 3"},{"comment":"The paper does not state whether code, trained models, or random seeds will be released; please add a reproducibility statement and a seed list.","section":"Reproducibility"},{"comment":"The repeated independent queries used to produce per-step frames are described clearly, but the text should emphasize more visibly that independently sampled frames are not constrained to be temporally consistent, a point the paper itself acknowledges only in the limitations section.","section":"§3.3 and Appendix B"}],"recommendation":"major_revision","confidential_remarks":"The paper's central claim is plausible and the theoretical motivation is sound, but the evidence in the main body is thinner than the abstract claims. The controlled ablation needs to be expanded to multiple tasks and seeds, and the main tables need a true one-step baseline. I do not see a novelty or attribution problem: the paper clearly distinguishes its contribution from ADM and from hierarchical latent world models. The requested changes are within the scope of a revision rather than a rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper's central claim—that long-horizon endpoint supervision, not the specific backbone, drives long-horizon prediction accuracy—is worth taking seriously. The controlled experiment of retraining ADM under the same long-horizon objective is exactly the right way to separate objective from architecture, and the fact that it closes much of the gap to DPWM is genuinely informative. That alone makes this more than another architecture paper.\n\nWhat's new: DPWM itself is a reasonable instantiation—non-recursive, endpoint-only, variable-horizon training—but the real contribution is the empirical demonstration that the training objective matters, plus the honest framing that the architecture is just an enabler. The Lipschitz error-bound derivation in App. A is standard but correctly framed as motivation, not proof.\n\nThe soft spots are real and concentrated where they matter. Table 2, the load-bearing ablation, is one task (humanoid_walk), no error bars, no seeds, no hyperparameter details for the K=100 ADM retraining. The paper asserts that ADM K=3 and K=100 share encoder and inference procedure, differing only in training horizon, but doesn't verify that the long-horizon retraining didn't require different tuning. If it did, the conclusion is confounded. Also, the main table (Table 1) lacks a short-horizon autoregressive baseline trained with K=1 and rolled out, which would be the natural comparison for the 'myopic world models' the title invokes. And the pixel benchmark in App. D compares DPWM K=100 vs ADM K=100—both long-horizon—so it doesn't test the objective-vs-backbone claim on pixels at all. No code or data release, though that's a fixable omission.\n\nNone of these are fatal. The central claim is plausible, and the paper is honest about its limitations. The missing baselines and error bars are exactly what a referee should ask for. I'd send this to review, with the expectation of major revisions. The paper is aimed at the world-model and model-based RL community, and the question it asks—should we train and evaluate at the temporal scale of use—is important enough to deserve referee time.\n\nRecommendation: send to peer review. Ask for error bars, a second task for the ADM retraining, a short-horizon baseline in Table 1, and a pixel-level short-horizon comparison.","headline":"Long-horizon endpoint supervision is a real and testable idea, and the ADM retraining is the right experiment, but the evidence is thinner than the abstract claims: one task, no error bars, and missing short-horizon baselines.","tokens_in":14075,"tokens_out":2764,"would_cite":true,"duration_ms":23610,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper argues that long-horizon world-model accuracy is driven by the training objective—end-to-end endpoint prediction—not by the choice of backbone architecture.","keywords":["world models","long-horizon prediction","direct endpoint prediction","non-recursive dynamics","compounding error","training objective","model-based reinforcement learning","end-to-end supervision"],"falsifier":"Reproduce Table 2 by training ADM at $K_{\\max}=100$ using exactly the published $K=3$ hyperparameters and inference procedure; if endpoint MSE at horizons 100 and 200 does not fall clearly below the $K=3$ numbers, the central claim that the training objective drives long-horizon accuracy is not supported. A complementary check is to train a one-step autoregressive world model with the same endpoint objective via backpropagation through time and see whether its long-horizon endpoint error drops to DPWM levels.","tokens_in":13049,"feed_emoji":"🎯","tokens_out":8493,"duration_ms":74839,"temperature":0.7,"pith_summary":"World models are usually trained to predict one step ahead and then rolled out recursively, so small errors compound over long horizons. This paper argues that long-horizon accuracy should be optimized directly: attach the loss to the endpoint observation reached after an entire action sequence, and train end-to-end. To make that practical, it introduces DPWM, a non-recursive model that compresses the action sequence into a single embedding and predicts the endpoint in one forward pass, so gradient depth stays fixed no matter how long the horizon. The central claim is that the training objective, rather than any particular backbone, is the main driver of long-horizon accuracy—supported by retraining a recurrent baseline under the same endpoint objective and seeing its gap to DPWM mostly close.","feed_headline":"The training objective, not the backbone, sets long-horizon accuracy","feed_subtitle":"A non-recursive world model predicts the final state in one pass; recurrent baselines close the gap when retrained the same way.","key_machinery":"The load-bearing object is the $K$-step endpoint prediction objective $\\mathcal{L}(\\theta) = \\mathbb{E}\\left[ \\| D(g(E(o_t), \\phi(a_{t:t+K-1}))) - o_{t+K}\\|^2 \\right]$ with $K$ sampled from a horizon distribution. The direct prediction model $g_\\theta$ is non-recursive: an action-sequence encoder $\\phi$ (a bidirectional windowed Transformer with rotary position embeddings, mean-pooled) turns the whole action list into one embedding, and a FiLM-conditioned residual MLP maps the initial observation latent to the endpoint latent in a single pass. Two mechanisms carry the argument: the error bound comparison (recursive rollout amplifies by $\\epsilon \\sum_{i=0}^{K-1} L^i$ while direct prediction pays only $\\delta_K$) and the gradient-alignment identity in Eq. (5), where the endpoint's parameter sensitivity weights each transition by the Jacobian product $J_{k\\to K}$ and the endpoint loss reproduces that weighting automatically. The architecture's role is enabler, not source of accuracy: it keeps gradient depth fixed so long-horizon end-to-end training is tractable.","core_discovery":"The paper's central claim is that long-horizon prediction accuracy in world models is largely determined by the training objective and prediction paradigm, not by the specific architecture. Formally, a $K$-step transition map $f^{(K)}$ can be approximated either by composing a one-step model $\\hat f$ and rolling it out, which gives an endpoint error bounded by $\\epsilon \\sum_{i=0}^{K-1} L^i$ and amplifies perturbations whenever the dynamics are expansive ($L>1$), or by learning a direct approximator $g_\\theta$ whose endpoint error is just $\\delta_K = \\sup \\|g_\\theta - f^{(K)}\\|$ with no recursive amplification. Training $g_\\theta$ with an endpoint loss over sampled horizons attaches gradients to the endpoint, so each transition is weighted by its downstream Jacobian influence exactly as in the sensitivity of the endpoint; the one-step objective weights all transitions uniformly and is misaligned with that sensitivity. DPWM instantiates this paradigm non-recursively—a Transformer action-sequence encoder plus a FiLM-conditioned dynamics module—so the gradient path from the loss to any parameter has fixed depth regardless of $K$. Consistent with the objective-first claim, retraining the recurrent baseline ADM with the same long-horizon endpoint objective substantially closes its gap to DPWM, while its published short-horizon training does poorly at long horizons.","pith_inferences":["An extension this paper leaves implicit is that the objective-first view predicts any sufficiently expressive backbone trained end-to-end at the same horizon will match DPWM; a direct test would fix the endpoint loss and vary only the sequence encoder.","A testable consequence for model-based RL is that evaluation should use endpoint error at the planning horizon rather than average rollout error, since the paper's own metric choice signals this shift.","The deterministic formulation does not cover stochastic dynamics; extending the endpoint objective to conditional distributions or conditional means is a natural next step this paper names as future work.","The non-recursive design also suggests a planning advantage: many candidate action sequences can be evaluated in parallel single forward passes, which the paper does not exploit."],"forward_implications":["If the objective is the main driver, world models should be trained and evaluated at the temporal horizon where they will actually be used; the paper shows $K_{\\max}=50$ degrades beyond its training horizon while $K_{\\max}=100$ stays accurate at $K=200$.","Recurrent or autoregressive models can also benefit: training them with an end-to-end endpoint objective (backpropagating through the unrolled rollout) should inherit the Jacobian-weighted gradient and improve long-horizon accuracy, as the ADM retraining experiment shows.","Direct endpoint prediction removes the recursive feedback path at inference, so compounding error from self-conditioning on predicted states is avoided even before considering the training objective.","A model trained only on endpoint supervision can still supply intermediate frames by querying successive action prefixes in separate forward passes, making it usable for trajectory-level planning without autoregressive rollout."],"supporting_citations":[{"why":"Supplies the Lipschitz compounding-error bound $\\epsilon \\sum_{i=0}^{K-1}L^i$ that motivates direct prediction over recursive rollout.","marker":"[2]"},{"why":"Defines the any-step dynamics model (ADM) used as the recurrent baseline and as the object of the retraining comparison that isolates objective from architecture.","marker":"[27, 28]"},{"why":"Provides the physics-prior baseline MoSim used to show that local physical fidelity does not by itself give long-horizon endpoint accuracy.","marker":"[21]"},{"why":"Supplies the expert policy checkpoint used to build the policy-distribution training and evaluation data for continuous control.","marker":"[20]"},{"why":"Defines the continuous-control benchmark tasks used in the state-based experiments.","marker":"[37]"},{"why":"Provides the Atari environment used for the pixel-based Pong experiment.","marker":"[4]"},{"why":"Documents the memory and optimization difficulty of backpropagating through many recurrent steps, the obstruction that the non-recursive architecture removes.","marker":"[32]"}],"fun_headline_variants":["Objective, not backbone, drives long-horizon world model accuracy","Training objective trumps architecture for long-horizon prediction","In world models, objective > architecture for long horizons","Long-horizon accuracy: objective over backbone"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the ADM $K=100$ versus $K=3$ comparison isolates the training horizon—same encoder, same inference procedure, stable optimization at $K=100$—so that if long-horizon training required different hyperparameters or changed inference, the conclusion that objective rather than backbone drives accuracy would be confounded.","fun_headline_variants_meta":{"raw":{"variants":["Objective, not backbone, drives long-horizon world model accuracy","Training objective trumps architecture for long-horizon prediction","In world models, objective > architecture for long horizons","Long-horizon accuracy: objective over backbone"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000722,"raw_usage":{"total_tokens":3334,"prompt_tokens":1135,"completion_tokens":2199,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":751,"completion_tokens_details":{"reasoning_tokens":2133}},"tokens_in":751,"tokens_out":2199,"duration_ms":14185,"temperature":1.0,"reasoning_tokens":2133,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T04:55:00.343671+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Reproduce Table 2 by training ADM at $K_{\\max}=100$ using exactly the published $K=3$ hyperparameters and inference procedure; if endpoint MSE at horizons 100 and 200 does not fall clearly below the $K=3$ numbers, the central claim that the training objective drives long-horizon accuracy is not supported. A complementary check is to train a one-step autoregressive world model with the same endpoint objective via backpropagation through time and see whether its long-horizon endpoint error drops to DPWM levels.","supporting_citations":[{"cited_title":"Neural motion simulator pushing the limit of world models in reinforcement learning","cited_arxiv_id":null,"evidence_quote":"Provides the physics-prior baseline MoSim used to show that local physical fidelity does not by itself give long-horizon endpoint accuracy."},{"cited_title":"dm_control: Software and tasks for continuous control.Software Impacts, 6:100022, 2020","cited_arxiv_id":null,"evidence_quote":"Defines the continuous-control benchmark tasks used in the state-based experiments."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the Atari environment used for the pixel-based Pong experiment."},{"cited_title":"On the difficulty of training recurrent neural networks","cited_arxiv_id":null,"evidence_quote":"Documents the memory and optimization difficulty of backpropagating through many recurrent steps, the obstruction that the non-recursive architecture removes."}],"review_version":1}