Pith. sign in

REVIEW 2 major objections 6 minor 54 references

DriftWorld claims that a drifting generative model can produce action-conditioned future frames in one forward pass, running 17x faster than diffusion world models at equal or better rollout quality, making large-scale action search and off

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 00:16 UTC pith:5P2LEISL

load-bearing objection Strong empirical case for single-step drifting world models, with a theory section that overclaims and timing that needs auditing; send to review. the 2 major comments →

arxiv 2607.15065 v1 pith:5P2LEISL submitted 2026-07-16 cs.RO cs.CVcs.LG

DriftWorld: Fast World Modeling through Drifting

Susie Lu , Haonan Chen , Weirui Ye , Yilun Du This is my paper
classification cs.RO cs.CVcs.LG
keywords drifting generative modelsaction-conditioned world modelsone-step video generationrobot manipulationinference-time planningoffline policy evaluationworld model speedgenerative video prediction
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

DriftWorld sets out to remove the inference-time bottleneck that limits robot world models. Its central claim is that drifting generative models, adapted to action-conditioned video prediction, can generate a chunk of future frames from the current observation and candidate actions in a single forward pass at 30+ fps, 17x faster on average than diffusion world models, without giving up rollout quality. The authors show this on five robotics benchmarks—Push-T, Robomimic, Bridge-V2, RT-1, and Language Table—matching or beating diffusion baselines on visual metrics, and then show the speed pays off twice: rolling out 50 candidate actions raises Push-T IoU from 0.635 to 0.781, and offline rollouts rank policies with up to 0.99 correlation to ground truth. If right, the main bottleneck on imagined rollouts for planning disappears.

Core claim

On the paper's own terms, the discovery is that a one-step generator is sufficient for high-fidelity action-conditioned world modeling. Instead of iteratively denoising at inference, DriftWorld learns an action-conditioned drift during training and regresses the generator toward the drifted target, moving the model's pushforward distribution onto the true conditional distribution of future frames. With the right adaptations—frame-wise action conditioning, a drifting loss measured in a pretrained visual feature space for real-world scenes, motion weighting to stop the model from collapsing to copying the previous frame, and action-accentuated negative samples—the single forward pass produces

What carries the argument

The drifting field V^{p,q}(x) = V^+_p(x) - V^-_q(x), a kernel-based mean-shift vector that attracts a generated video chunk toward the single ground-truth future chunk and repels it from N_neg self-generated negative chunks. It is computed per spatial location in either pixel space or a pretrained visual feature space and aggregated over multiple kernel temperatures; training is a fixed-point regression toward the stop-gradient drifted sample. Three additions carry the action-conditioned adaptation: frame-wise action conditioning in the U-Net, motion weighting that upweights moving regions of the feature map, and an action-accentuated negative distribution that mixes generated futures with r

Load-bearing premise

That the drift field estimated from one ground-truth future chunk and a set of self-generated negatives accurately measures how far the generated distribution is from the true action-conditioned future distribution; if that estimate is biased or too noisy, the one-step generator can settle into copying the last observed frame instead of following the action.

What would settle it

Train DriftWorld with the same objective but replace the ground-truth positive future chunk with a randomly chosen future chunk from a different trajectory; if rollout visual quality and action-following metrics are unchanged, the positive sample is not carrying the conditioning signal and the drifting loss is not the source of the reported gains.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • A planner can evaluate many more proposals per control cycle: each rollout is one forward pass at 30+ fps, compared with 0.03–3.2 s/frame for diffusion baselines.
  • Visual rollout quality matches or exceeds baselines on Push-T, Robomimic, Bridge-V2, RT-1, and Language Table across SSIM, PSNR, LPIPS, FID, and FVD.
  • Inference-time action search over 50 proposals lifts Push-T IoU from 0.635 to 0.781, beating diffusion world models at a fraction of the runtime.
  • Offline, rolling out policies in DriftWorld ranks them with 0.9515–0.9916 Pearson correlation to ground-truth performance, enabling hardware-free policy screening.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the single-positive-sample drift estimator generalizes to any conditional generative task with exactly one correct continuation, such as instruction-conditioned video, where diffusion-based alternatives still pay a sampling tax.
  • Beyond the paper: at 30+ fps, closed-loop model-predictive control with hundreds of sampled futures becomes plausible at deployment time, not just in offline analysis.
  • Beyond the paper: the 0.95–0.99 ranking correlations suggest using DriftWorld as a cheap pretest filter—screen many policy checkpoints or hyperparameters in simulation and deploy only the survivors.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper introduces DriftWorld, an action-conditioned world model for robot manipulation that generates future frames in a single forward pass by training a U-Net with a drifting loss. The model uses one ground-truth future chunk as the positive sample and multiple self-generated chunks as negatives, with feature-space drift (DINOv2/v3), motion weighting, and self-forcing as key components. The authors report that DriftWorld matches or exceeds diffusion-based world-model baselines in image/video quality metrics on Push-T, Robomimic, Bridge-V2, RT-1, and Language Table while being on average 17x faster, that GPC-RANK with K=50 improves Push-T IoU from 0.635 to 0.781, and that offline policy rankings correlate with ground truth up to 0.99.

Significance. If the empirical results hold, DriftWorld is a practically important contribution: it demonstrates that a single-step world model can compete with much slower diffusion-based world models on standard benchmarks, and the speed advantage is directly leveraged for inference-time planning and offline policy evaluation. The paper includes a clean MSE-baseline control (same U-Net, one step, pixel MSE) that isolates the drifting loss, and the ablations in Table 5 show that feature-space drift, motion weighting, and self-forcing each contribute. The code link is provided. However, the theoretical justification in Section 3.2 is incorrect as stated, and the Robomimic policy-ranking results in Section 4.4 may be affected by circular post-training. These issues are substantial enough that the manuscript needs revision before the claims can be accepted at face value.

major comments (2)
  1. [Section 3.2] The claim that 'equilibrium is reached when the generated distribution q_i matches the true conditional video distribution p' is false with a single positive sample. With one positive sample y+, the kernel cancels and V+(x) = y+ - x, so E[V+(x)] = E_p[y] - x, which is not zero at a typical x ~ p. At q = p, V- is the kernel mean-shift of p toward the nearest mode. Concrete counterexample: p = 0.5*delta_{-10} + 0.5*delta_{10} with a localized kernel at x = -10 gives V+ approx 10 and V- approx 0, so V approx 10 != 0. Thus p is not a fixed point of the drift map. This is load-bearing because it justifies the training objective. Please provide a corrected fixed-point analysis or explicitly present the method as heuristic.
  2. [Section 4.4 / Appendix D.3] The Robomimic policy-ranking correlations (0.9916, 0.9250) may be inflated by circular evaluation. The paper says the model is post-trained on failure demonstrations 'created by rolling out early checkpoints of a diffusion policy,' and the evaluated policies are exactly such checkpoints (Lift epochs 2,4,...,18; Can epochs 50,...,400 plus 75). If the post-training data comes from the same checkpoint epochs that are later evaluated, this is test-set contamination. Please clarify the overlap, and report correlations using held-out policies or without post-training.
minor comments (6)
  1. [Section 3.2 vs Algorithm 1] The text says negative samples are model-generated chunks, but Algorithm 1 includes obs[-1] in the negative set. Clarify the composition of the negative set and how it relates to the 'no-action distribution' mixture in Section 3.3.
  2. [Section 3.3 / Appendix C.2] The accentuation scale alpha is used in Figure 13 and Appendix C.2 but is never defined in the main text. Define alpha when the action-accentuation mixture is introduced.
  3. [Section 4.2] 'Across all four environments' should be 'across all five datasets' (Push-T, Robomimic, Bridge-V2, RT-1, Language Table).
  4. [Table 1] Clarify how 'seconds per generated frame' is measured for multi-frame chunks, and why the full-episode timing differs from the 64-frame timing for the same model.
  5. [Table 1] The baseline 'A VDC' is not spelled out at first use; state what the acronym stands for.
  6. [Notation] The history length is F in Sections 3.1 and 3.3 but H in Section 3.4; use one symbol consistently.

Circularity Check

0 steps flagged

No circularity found: DriftWorld's drift field, fixed-point loss, and validation benchmarks are self-contained and externally compared.

full rationale

DriftWorld's central derivation is presented directly in the paper: the drift field V = V+ - V- in Sec. 3.2, the fixed-point training loss in Sec. 3.3, the action-conditioned U-Net in Sec. 3.4, and single-pass inference in Sec. 3.5. The headline predictions (SSIM/PSNR/LPIPS/FID/FVD, Push-T IoU gains, and policy-ranking correlations) are measured on validation data against external baselines (GPC, Ctrl-World, IRASim, VDM, LVDM) plus a purpose-built MSE baseline that shares the U-Net backbone but not the drifting loss. Thus the quality gains are not a fitted constant relabeled as a prediction. The paper cites the authors' prior drifting paper [6] and GPC [5], and these are foundational references, but the load-bearing argument does not reduce to them: the training equations are written out, no uniqueness theorem is imported, no ansatz is adopted solely by citation, and the empirical claims are externally falsifiable. The Sec. 3.2 assertion that equilibrium is reached when q = p is an unproven convergence/consistency statement; with a single positive sample the realized drift is noisy and the fixed point may not be exactly p. That is a theoretical correctness risk, not a case where a prediction is equivalent to an input by construction. No circular step meeting the evidentiary bar was found.

Axiom & Free-Parameter Ledger

6 free parameters · 4 axioms · 0 invented entities

The core method introduces no free constants fitted to evaluation results; the listed items are training hyperparameters chosen per dataset, which is standard practice but means several of the central design choices (N_neg, tau, alpha, gamma, motion-weighting scalars, T) are hand-tuned rather than derived. The most fragile pieces are the equilibrium assumption for the single-positive-sample conditional drift (asserted, not derived) and the reliance on DINOv3 features for the loss geometry. The Robomimic policy-eval correlations additionally rely on post-training on failure demonstrations ("we post-train the models on a dataset with a large percentage of failure demonstrations," Appendix D.3), which is load-bearing for the offline-evaluation claim.

free parameters (6)
  • Number of negative samples N_neg per forward pass = 8 (Push-T), 32 (Robomimic), 64 (Bridge-V2/RT-1/Language Table)
    Controls the estimator variance of the repulsive drift term and training memory; chosen per dataset (Table 8). High memory use is admitted as a limitation in Section 5.
  • Kernel temperatures tau (multi-temperature aggregation) = {0.02, 0.05, 0.2} or {0.02, 0.05} per dataset
    Sets the attraction/repulsion radius of the drift kernel k(x,y)=exp(-||x-y||/(tau*sqrt(C))) (Appendix D); per-dataset choices affect which samples are treated as near neighbors.
  • Action-accentuation scale alpha = Sampled log-uniform over [1,4] during training; tested 1.0-3.5 at inference
    The U-Net takes alpha as input and Appendix C.2 shows gripper motion is blurry at low alpha; the chosen range directly affects action-following quality.
  • Motion-weighting scalars lambda, alpha_mw = c_{h,w} = 1 + lambda*tanh(alpha_mw * n_{h,w}); numeric values not reported
    Spatially reweights the drifting loss toward moving regions to prevent identity-mapping collapse (Section 3.3, Fig 14); essential for real-robot autoregressive generation, but exact values are unspecified.
  • Prediction horizon T per dataset = 4 (Push-T), 2 (Robomimic), 1 (Bridge-V2, RT-1, Language Table)
    Chunk length per forward pass; Table C.1 shows T=4 is best on Push-T; affects speed, autoregressive drift, and action conditioning.
  • No-action mixture weight gamma = Not reported numerically (defined gamma in [0,1))
    Mixes real no-action frames into the negative distribution to accentuate action following (Section 3.3); the value is unspecified, leaving the action-accentuation strength uncalibrated in the write-up.
axioms (4)
  • ad hoc to paper The conditional drifting field V_{p,q}(x)=V+_p(x)-V-_q(x), estimated with a single positive sample and N_neg self-generated negatives, is a consistent estimator of the displacement that moves the pushforward q to p(obs | action, history), and the fixed-point MSE training converges to that equilibriu
    Asserted in Section 3.2 ("equilibrium is reached when the generated distribution q_i matches the true conditional video distribution p") with no proof; this single-positive-sample conditional variant is new relative to drifting [6].
  • domain assumption DINOv2/v3 feature space (and SD3 VAE latent space) provides a metric in which kernel attraction/repulsion drives samples toward perceptually and semantically correct futures.
    Feature-space drift is load-bearing for real-robot datasets (Section 3.3, Table 5: FVD 168.34 to 6.20); the choice is validated only empirically, and the dependency on a robust pretrained extractor is acknowledged as a limitation in Section 5.
  • domain assumption Training/evaluation distributions match: world-model quality is measured on validation sets of the same five benchmarks, and the learned drift generalizes to the policy-rollout distribution (with failure demos added via post-training for Robomimic).
    Standard benchmark assumption (Section 4.1); for Robomimic policy evaluation the distribution is actively repaired by post-training on failures (Appendix D.3), which is disclosed but means the 0.99/0.93 correlations describe a calibrated simulator.
  • standard math Background mathematics of kernel mean-shift / fixed-point iteration, and the correctness of DINOv3/SD3-VAE features as used.
    The drift computation and normalizations (Appendix D) rely on standard kernel/k-means-shift machinery; no formal proof accompanies the paper.

pith-pipeline@v1.3.0-alltime-deepseek · 23475 in / 26237 out tokens · 267717 ms · 2026-08-02T00:16:03.072563+00:00 · methodology

0 comments
read the original abstract

Predictive world models enable robots to plan by imagining the outcomes of their actions, but their value for control hinges on generating many rollouts quickly. This creates a bottleneck for diffusion-based world models: multistep sampling makes each rollout expensive, limiting large-scale action search at inference time. We introduce DriftWorld, an action-conditioned world model based on drifting generative models. Rather than denoising iteratively at inference, DriftWorld learns an action-conditioned drift during training, allowing it to generate future frames from the current observation and a candidate action sequence in a single forward pass at 30+ fps, which is 17x faster on average than diffusion based baselines. We evaluate DriftWorld on standard vision-based robotic manipulation benchmarks, including Bridge-V2, RT-1, Language Table, Push-T, and Robomimic. By producing rollouts that are both accurate and fast, DriftWorld achieves state-of-the-art decision-making performance with far less inference time than diffusion-based world model baselines. Beyond online control, DriftWorld can also serve as an offline simulator for ranking real-world robot policies, with rollout-based scores correlating with ground truth at up to 0.99. These results show that drifting models are a strong fit for robot world modeling, where fast, high-quality imagination directly supports planning and policy evaluation.

Figures

Figures reproduced from arXiv: 2607.15065 by Haonan Chen, Susie Lu, Weirui Ye, Yilun Du.

Figure 1
Figure 1. Figure 1: Overview of DriftWorld. DriftWorld is an action-conditioned world model based on drifting that generates future frames in a single forward pass. It achieves 30+ fps generation, which is significantly faster than existing models on all five environments. DriftWorld can be used for high-quality generation, efficient planning, and offline simulation of policies. field defined by the initial observation and ac… view at source ↗
Figure 2
Figure 2. Figure 2: Drifting Architecture and Training Overview. (a) We use a U-Net architecture conditioned on history observations and robot actions. (b) DriftWorld is trained using a drifting loss, which is a contrastive loss that encourages the generated future frames (negative samples) to drift toward the ground-truth future frames (the positive sample). Accentuating Action Following. To increase the model’s adherence to… view at source ↗
Figure 3
Figure 3. Figure 3: Comparison of different world models’ rollouts for Push-T. The first row is the ground truth, and the remaining three rows are autoregressively generated rollouts with a length of 140 frames. DriftWorld’s rollout matches the ground truth. In contrast, for the GPC diffusion world model and the MSE baseline, the target is partially wiped out in the later frames, leading to an incorrect final position for the… view at source ↗
Figure 4
Figure 4. Figure 4: Comparison of different world models’ rollouts for Bridge-V2. The first row is the ground truth, and the remaining rows are generated rollouts. DriftWorld accurately simulates contact interactions of the robot gripper with the cucumber, plate, and mushroom (left) and block (right). In contrast, IRASim’s video displays inaccurate gripper motions, and WorldGym’s video exhibits artifacts on the plate and bloc… view at source ↗
Figure 5
Figure 5. Figure 5: Comparison of different world models’ rollouts for RT-1. The first row is the ground truth, and the remaining rows are generated rollouts. DriftWorld’s generated video displays the correct motion of picking up the bar (left) and opening the drawer (right). 8 [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Policy Evaluation. On the Lift, Can, and Push-T tasks, DriftWorld achieves a higher correlation coefficient between predicted and ground-truth values of the success rates or IoU scores than the baseline. 9 [PITH_FULL_IMAGE:figures/full_fig_p009_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Visualization of ablation on fea￾ture space. With the DINOv2 or DINOv3 feature extractor, the generated robot gripper is sharp. Without any features, the gripper is blurry. Ablation on feature space. The representation space in which the drifting loss is computed significantly influ￾ences generation quality. On the Push-T and Robomimic environments, computing the drifting loss directly in pixel space is su… view at source ↗
Figure 8
Figure 8. Figure 8: Comparison of videos generated by DriftWorld vs. baselines on Language Table. GT WorldGym IRASim DriftWorld (Ours) [PITH_FULL_IMAGE:figures/full_fig_p015_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Comparison of videos generated by DriftWorld vs. baselines on Bridge-V2. 15 [PITH_FULL_IMAGE:figures/full_fig_p015_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Comparison of videos generated by DriftWorld vs. baselines on RT-1. A.2 Comparison of DriftWorld’s Policy Rollouts As Figures 11 and 12 show, policy rollouts in DriftWorld are very similar to the ground-truth rollouts. GT Example 1 Example 2 GPC Diffusion DriftWorld (Ours) [PITH_FULL_IMAGE:figures/full_fig_p016_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Policy Rollouts for Push-T. Qualitative comparison showing that DriftWorld closely matches ground-truth rollouts and adheres to the action conditioning significantly more accurately than the baseline GPC diffusion model does. B Additional Quantitative Results [PITH_FULL_IMAGE:figures/full_fig_p016_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Policy Rollouts for the Robomimic Lift, Can, and Square Tasks. The generated videos closely match the ground truth on all three tasks. C Additional Ablations for DriftWorld In this section, we conduct further ablations of core components of DriftWorld. C.1 Single-Frame versus Chunk-Level Simulation DriftWorld supports both single-frame and chunk-level simulation in a single forward pass. Gener￾ating 4 fut… view at source ↗
Figure 13
Figure 13. Figure 13: Comparison of DriftWorld’s generated videos on Bridge-V2 under different scales α for ac￾centuating action following. As α increases, the generated gripper follows the specified action more closely. See the 2nd, 3rd, and 6th frames. GT No Motion Weight Motion Weight [PITH_FULL_IMAGE:figures/full_fig_p018_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Effect of motion weighting on DriftWorld’s autoregressive generation on RT-1. Without motion weighting, the robot gripper remains mostly stationary in later frames. In contrast, applying motion weighting ensures the gripper correctly moves according to the action conditioning. D Implementation Details D.1 Details for DriftWorld The hyperparameters of DriftWorld are given in [PITH_FULL_IMAGE:figures/full_… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

54 extracted references · 16 linked inside Pith

  1. [1]

    S. Yang, Y . Du, S. K. S. Ghasemipour, J. Tompson, L. P. Kaelbling, D. Schuurmans, and P. Abbeel. Learning interactive real-world simulators. InThe Twelfth International Conference on Learning Representations, 2024

    2024

  2. [2]

    F. Zhu, H. Wu, S. Guo, Y . Liu, C. Cheang, and T. Kong. IraSim: A fine-grained world model for robot manipulation. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 9834–9844, 2025

    2025

  3. [3]

    Y . Guo, L. X. Shi, J. Chen, and C. Finn. Ctrl-World: A controllable generative world model for robot manipulation. InICLR, 2026

    2026

  4. [4]

    Y . Du, S. Yang, P. Florence, F. Xia, A. Wahid, P. Sermanet, T. Yu, P. Abbeel, J. B. Tenen- baum, L. Kaelbling, et al. Video language planning. InInternational Conference on Learning Representations, volume 2024, pages 31138–31155, 2024

    2024

  5. [5]

    H. Qi, H. Yin, A. Zhu, Y . Du, and H. Yang. Inference-time enhancement of generative robot policies via predictive world modeling.IEEE Robotics and Automation Letters, 2026

    2026

  6. [6]

    M. Deng, H. Li, T. Li, Y . Du, and K. He. Generative modeling via drifting.arXiv preprint arXiv:2602.04770, 2026

    Pith/arXiv arXiv 2026

  7. [7]

    Oquab, T

    M. Oquab, T. Darcet, T. Moutakanni, H. V o, M. Szafraniec, V . Khalidov, P. Fernandez, D. Haz- iza, F. Massa, A. El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023

    Pith/arXiv arXiv 2023

  8. [8]

    Siméoni, H

    O. Siméoni, H. V . V o, M. Seitzer, F. Baldassarre, M. Oquab, C. Jose, V . Khalidov, M. Szafraniec, S. Yi, M. Ramamonjisoa, et al. DINOv3.arXiv preprint arXiv:2508.10104, 2025

    Pith/arXiv arXiv 2025

  9. [9]

    H. R. Walke, K. Black, T. Z. Zhao, Q. Vuong, C. Zheng, P. Hansen-Estruch, A. W. He, V . My- ers, M. J. Kim, M. Du, et al. BridgeData V2: A dataset for robot learning at scale. InConfer- ence on Robot Learning, pages 1723–1736. PMLR, 2023

    2023

  10. [10]

    Brohan, N

    A. Brohan, N. Brown, J. Carbajal, Y . Chebotar, J. Dabis, C. Finn, K. Gopalakrishnan, K. Haus- man, A. Herzog, J. Hsu, J. Ibarz, B. Ichter, A. Irpan, T. Jackson, S. Jesmonth, N. Joshi, R. Ju- lian, D. Kalashnikov, Y . Kuang, I. Leal, K.-H. Lee, S. Levine, Y . Lu, U. Malla, D. Manjunath, I. Mordatch, O. Nachum, C. Parada, J. Peralta, E. Perez, K. Pertsch, ...

    Pith/arXiv arXiv 2022

  11. [11]

    Lynch, A

    C. Lynch, A. Wahid, J. Tompson, T. Ding, J. Betker, R. Baruch, T. Armstrong, and P. Florence. Interactive language: Talking to robots in real time.IEEE Robotics and Automation Letters, 2023

    2023

  12. [12]

    Florence, C

    P. Florence, C. Lynch, A. Zeng, O. A. Ramirez, A. Wahid, L. Downs, A. Wong, J. Lee, I. Mor- datch, and J. Tompson. Implicit behavioral cloning. InConference on robot learning, pages 158–168. PMLR, 2022

    2022

  13. [13]

    C. Chi, Z. Xu, S. Feng, E. Cousineau, Y . Du, B. Burchfiel, R. Tedrake, and S. Song. Diffusion policy: Visuomotor policy learning via action diffusion.The International Journal of Robotics Research, 44(10-11):1684–1704, 2025

    2025

  14. [14]

    Mandlekar, D

    A. Mandlekar, D. Xu, J. Wong, S. Nasiriany, C. Wang, R. Kulkarni, L. Fei-Fei, S. Savarese, Y . Zhu, and R. Martín-Martín. What matters in learning from offline human demonstrations for robot manipulation.arXiv preprint arXiv:2108.03298, 2021. 11

    Pith/arXiv arXiv 2021

  15. [15]

    Ha and J

    D. Ha and J. Schmidhuber. World models.arXiv preprint arXiv:1803.10122, 2018

    Pith/arXiv arXiv 2018

  16. [16]

    Hafner, T

    D. Hafner, T. Lillicrap, J. Ba, and M. Norouzi. Dream to control: Learning behaviors by latent imagination.arXiv preprint arXiv:1912.01603, 2019

    Pith/arXiv arXiv 1912

  17. [17]

    Lozano-Perez

    T. Lozano-Perez. Robot programming.Proceedings of the IEEE, 71(7):821–841, 1983

    1983

  18. [18]

    B. Hou, G. Li, J. Jia, T. An, X. Guo, S. Leng, H. Geng, Y . Ze, T. Harada, P. Torr, et al. World model for robot learning: A comprehensive survey.arXiv preprint arXiv:2605.00080, 2026

    Pith/arXiv arXiv 2026

  19. [19]

    H. Chen, Y . Niu, K. Hong, S. Liu, Y . Wang, Y . Li, and K. R. Driggs-Campbell. Predicting object interactions with behavior primitives: An application in stowing tasks. In7th Annual Conference on Robot Learning, 2023

    2023

  20. [20]

    K. Hong, H. Chen, J. Xu, R. Wang, K. Wang, M. Zhang, S. Liu, Y . Zhu, Y . Li, and K. Driggs- Campbell. Gotta Scoop ’Em All: Sim-and-Real Co-Training of Graph-Based Neural Dynamics for Long-Horizon Scooping. InIEEE International Conference on Robotics and Automation (ICRA), 2026. To appear

    2026

  21. [21]

    A. Hu, L. Russell, H. Yeo, Z. Murez, G. Fedoseev, A. Kendall, J. Shotton, and G. Corrado. Gaia-1: A generative world model for autonomous driving, 2023

    2023

  22. [22]

    Bruce, M

    J. Bruce, M. Dennis, A. Edwards, J. Parker-Holder, Y . Shi, E. Hughes, M. Lai, A. Mavalankar, R. Steigerwald, C. Apps, Y . Aytar, S. Bechtle, F. Behbahani, S. Chan, N. Heess, L. Gonzalez, S. Osindero, S. Ozair, S. Reed, J. Zhang, K. Zolna, J. Clune, N. de Freitas, S. Singh, and T. Rocktäschel. Genie: Generative interactive environments. InInternational Co...

    2024

  23. [23]

    Cosmos World Foundation Model Platform for Physical AI, 2025

    NVIDIA. Cosmos World Foundation Model Platform for Physical AI, 2025

    2025

  24. [24]

    B. Chen, T. Zhang, H. Geng, C. Zhang, P. Li, K. Song, W. T. Freeman, J. Malik, P. Abbeel, R. Tedrake, et al. Large video planner enables generalizable robot control.arXiv preprint arXiv:2512.15840, 2025

    Pith/arXiv arXiv 2025

  25. [25]

    H. He, Y . Zhang, L. Lin, Z. Xu, and L. Pan. Pre-trained video generative models as world simulators. InProceedings of the AAAI Conference on Artificial Intelligence, 2026

    2026

  26. [26]

    Rigter, T

    M. Rigter, T. Gupta, A. Hilmkil, and C. Ma. A VID: Adapting video diffusion models to world models.Reinforcement Learning Journal, 6:737–764, 2025

    2025

  27. [27]

    Huang, J

    S. Huang, J. Wu, Q. Zhou, S. Miao, and M. Long. Vid2World: Crafting video diffusion models to interactive world models. InInternational Conference on Learning Representations (ICLR), 2026

    2026

  28. [28]

    G. R. Team, K. Choromanski, C. Devin, Y . Du, D. Dwibedi, R. Gao, A. Jindal, T. Kipf, S. Kir- mani, I. Leal, et al. Evaluating gemini robotics policies in a veo world simulator.arXiv preprint arXiv:2512.10675, 2025

    arXiv 2025

  29. [29]

    C. Zhu, R. Yu, S. Feng, B. Burchfiel, P. Shah, and A. Gupta. Unified world models: Coupling video and action diffusion for pretraining on large robotic datasets. InProceedings of Robotics: Science and Systems (RSS), 2025

    2025

  30. [30]

    Quevedo, A

    J. Quevedo, A. K. Sharma, Y . Sun, V . Suryavanshi, P. Liang, and S. Yang. Worldgym: World model as an environment for policy evaluation.arXiv preprint arXiv:2506.00613, 2025

    arXiv 2025

  31. [31]

    Y . Wang, R. Syed, F. Wu, M. Zhang, A. Onol, J. Barreiros, H. Nayyeri, T. Dear, H. Zhang, and Y . Li. Interactive world simulator for robot policy training and evaluation. InRobotics: Science and Systems (RSS), 2026

    2026

  32. [32]

    Salimans and J

    T. Salimans and J. Ho. Progressive distillation for fast sampling of diffusion models, 2022. 12

    2022

  33. [33]

    Y . Song, P. Dhariwal, M. Chen, and I. Sutskever. Consistency models. InICML, 2023

    2023

  34. [34]

    Song and P

    Y . Song and P. Dhariwal. Improved techniques for training consistency models. InInterna- tional Conference on Learning Representations, volume 2024, pages 15078–15097, 2024

    2024

  35. [35]

    X. Liu, C. Gong, and Q. Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow.arXiv preprint arXiv:2209.03003, 2022

    Pith/arXiv arXiv 2022

  36. [36]

    Sauer, D

    A. Sauer, D. Lorenz, A. Blattmann, and R. Rombach. Adversarial diffusion distillation. In ECCV 2024, page 87–103, 2024

    2024

  37. [37]

    R. Gao, K. Chen, B. Xiao, L. Hong, Z. Li, and Q. Xu. MagicDrive-V2: High-resolution long video generation for autonomous driving with adaptive control. InProceedings of the IEEE/CVF International Conference on Computer Vision, 2025

    2025

  38. [38]

    Lin and X

    S. Lin and X. Yang. Animatediff-lightning: Cross-model diffusion distillation.arXiv preprint arXiv:2403.12706, 2024

    Pith/arXiv arXiv 2024

  39. [39]

    Hansen, X

    N. Hansen, X. Wang, and H. Su. Temporal difference learning for model predictive control. arXiv preprint arXiv:2203.04955, 2022

    Pith/arXiv arXiv 2022

  40. [40]

    Y . Du, S. Yang, B. Dai, H. Dai, O. Nachum, J. Tenenbaum, D. Schuurmans, and P. Abbeel. Learning universal policies via text-guided video generation.Advances in Neural Information Processing Systems, 36:9156–9172, 2023

    2023

  41. [41]

    M. J. Kim, Y . Gao, T.-Y . Lin, Y .-C. Lin, Y . Ge, G. Lam, P. Liang, S. Song, M.-Y . Liu, C. Finn, and J. Gu. Cosmos policy: Fine-tuning video models for visuomotor control and planning. arXiv preprint arXiv:2601.16163, 2026

    Pith/arXiv arXiv 2026

  42. [42]

    S. Ye, Y . Ge, K. Zheng, S. Gao, S. Yu, G. Kurian, S. Indupuru, Y . L. Tan, C. Zhu, J. Xiang, et al. World action models are zero-shot policies.arXiv preprint arXiv:2602.15922, 2026

    Pith/arXiv arXiv 2026

  43. [43]

    P.-C. Ko, J. Mao, Y . Du, S.-H. Sun, and J. B. Tenenbaum. Learning to act from actionless videos through dense correspondences. InInternational Conference on Learning Representa- tions, volume 2024, pages 40938–40958, 2024

    2024

  44. [44]

    Esser, S

    P. Esser, S. Kulal, A. Blattmann, R. Entezari, J. Müller, H. Saini, Y . Levi, D. Lorenz, A. Sauer, F. Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In ICML, 2024

    2024

  45. [45]

    Rombach, A

    R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer. High-resolution image syn- thesis with latent diffusion models. InCVPR, pages 10684–10695, 2022

    2022

  46. [46]

    J. Ho, T. Salimans, A. Gritsenko, W. Chan, M. Norouzi, and D. J. Fleet. Video diffusion models.Advances in Neural Information Processing Systems, 35:8633–8646, 2022

    2022

  47. [47]

    Y . He, T. Yang, Y . Zhang, Y . Shan, and Q. Chen. Latent video diffusion models for high-fidelity long video generation.arXiv preprint arXiv:2211.13221, 2022

    Pith/arXiv arXiv 2022

  48. [48]

    Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli. Image quality assessment: from error visibility to structural similarity.IEEE Transactions on Image Processing, 13(4):600– 612, 2004

    2004

  49. [49]

    Hore and D

    A. Hore and D. Ziou. Image quality metrics: PSNR vs. SSIM. In20th International Conference on Pattern Recognition, pages 2366–2369. IEEE, 2010

    2010

  50. [50]

    Zhang, P

    R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang. The unreasonable effectiveness of deep features as a perceptual metric. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 586–595, 2018. 13

    2018

  51. [51]

    Heusel, H

    M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium.Advances in Neural Information Processing Systems, 30, 2017

    2017

  52. [52]

    Unterthiner, S

    T. Unterthiner, S. Van Steenkiste, K. Kurach, R. Marinier, M. Michalski, and S. Gelly. To- wards accurate generative models of video: A new metric & challenges.arXiv preprint arXiv:1812.01717, 2018

    Pith/arXiv arXiv 2018

  53. [53]

    Huang, Z

    X. Huang, Z. Li, G. He, M. Zhou, and E. Shechtman. Self forcing: Bridging the train-test gap in autoregressive video diffusion.Advances in Neural Information Processing Systems, 38: 167283–167308, 2025

    2025

  54. [54]

    J. Ho, A. Jain, and P. Abbeel. Denoising diffusion probabilistic models.Advances in Neural Information Processing Systems, 33:6840–6851, 2020. 14 Appendix Section A presents additional qualitative results: (i) visualizations of DriftWorld’s generated videos on Bridge-V2, RT-1, and Language Table and (ii) visualizations of policy rollouts in DriftWorld. Se...

    arXiv 2020